New GML variable scope rules will break old code

[Update: Please read the comments from YoYoGames CTO Russell Kay, at the bottom of this article. As it turns out, the implications of the changes that I expressed concerns about in this article were overblown. The YoYoGames tech blog article that caused my concerns wasn’t clear enough in describing them, resulting in my misunderstanding of the severity of the changes.]

Today’s YoYoGames tech blog deals with GML variable scope rules. I was dismayed to read that they are changing the scoping rules, which will result in old code breaking.

I’m normally very supportive of the decisions YYG has made with the development of GameMaker, particularly in the GM:S era, but this is probably the single worst thing that I’ve read about the development of GameMaker since I started using it in 2010.

I have a lot to say about this. First, I’d like to address the specific changes. Then I’ll talk a bit on the philosophy of how I would like YYG to treat me as a developer who relies on their tools.

Local scope now Event-wide, no longer script-wide

From the tech blog article:

  • Local Scope – if a variable is declared using the var keyword then it is only visible within this script or event and it is at local scope.

[…]

Some gotchas with Studio scoping

Local scope is local across the whole event, even if the variables are within different code actions – this can be confusing coming from earlier versions of GameMaker where variables are local to actions (not the whole event). This is due to optimisations within Studio that concatenates all the actions together and turns DnD into code, so the whole event becomes a single script that is executed.

I’d like to point out: it’s not just confusing coming from an earlier version of GameMaker — it’s confusing for newbies who have never programmed before, as well.

It’s not confusing just because they changed the way it works — it’s also confusing because now you have to be conscious of any and all other variables that may have been instantiated in any other action that is happening during the Event.

It means that I now need to be cognizant of every other Action that belongs to the Event, when I’m looking at any one Action. This makes it harder to understand any one Action — since the Actions aren’t isolated any longer, you need to look at the rest of the Event now, to see if there are any more references to the locally declared var in this Action. This makes the code harder to debug.

If I write a script() and call it in an action inside of one of my Object’s Events, I want the things that are local to that script to be isolated to that script — not be subject to things happening outside of it in the other Actions that may be happening in that Event. To me, that’s just proper encapsulation.

Under the new scope rules, if I want to ensure that my local variables are really being treated as such, I have to come up with unique names for each of them, so that there will not be a possible conflict if I happen to call another script that has a similar local variable.

For example, instead of being able to write:

script1{var a, b;}

script2{var a, b;}

…I now have to write…

script2{var c, d;}

…in order to ensure that variables that should be local to script2 are never in conflict with the local variables of script1.

That is an insane burden to place on a programmer, to come up with so many unique local var names, and keep track of them. When coding in a script, I should not have to think about any other script in my project.

And for how much of a performance optimization, exactly? What are we gaining from this pain? How can YYG think it’s worth it? A good programming language reduces the cognitive burden placed on the programmer. This increases it.

Globalvar is bad, so we’re going to take it away

From the Tech Blog:

Globalvar keyword can hide local and instance variables, if the same name is used for a globalvar variable as one used for local and instance variables then strange things can occur, the strangeness will be different between HTML5 and YYC as well as the JavaScript and C++ languages have different ideas on how scoping works so there can be some undefined results. We will be introducing an error to ensure that this is disallowed in future versions, but it is a bad idea and something you should be wary of. (The whole globalvar keyword is a bad idea ™ and will be removed in a future version of GML, but for now we have left it in for backward compatibility and bad habits).

In GML, globalvar is syntactic sugar. It allows the developer from having to type “global.variable” every time they want to reference a global variable.

It’s true that if you declare a global variable using globalvar, doing so will cause instance and local variables that have the same name inaccessible. That’s a mistake that can happen when you program and don’t understand scope. That doesn’t justify removing global scope from GML, though.

As far as the argument that “global variables are bad” — true, it can cause problems as we just mentioned. Sure, global variables can be overused, and misused, often by newbies and bad programmers.

True, they often are not needed. Certainly, a seasoned programmer will learn with experience that global variables are very often not necessary, and that there are other ways to accomplish a task that a newbie programmer will solve with a global variable.

But there are times when a global variable is expedient and using it makes code simpler to write and understand.

As well, even as widely discouraged as global scope is among professional programmers, global scope has not been removed from other languages once they were introduced. Their use has been discouraged, but the feature remains in languages to support existing code, and it is not always a mistake to use a feature of a language, even though the feature might lend itself to misuse.

So should it be with GML. The cost of breaking legacy code that relies on globalvar is not worth the payoff of enforcing “purity” of global-free code.

Using object_id.variable vs. instance.variable

From the Tech Blog:

Using an object name or id as a prefix will work differently across different platforms and also work differently if the variable is being read or written to, so it is recommended that you do not do this, it is better (and more sensible to use the with syntax rather than an object prefix). So do not do this

objPlayer.x = 5;
or 
objPlayer.x = enemy.x;

As it will do different things on different platforms.

It is much better to do this

with( objPlayer ) {
	x = enemy.x;
}

as this will work consistently across all target platforms.

Being able to call object.variable in GML is a unique feature to GML. It is very different from other object-oriented languages like Java or C#, in which code may only reference an instantiated instance of a class, not a class directly.

In GML, by contrast, it has long been possible to reference the object by the object_id rather than a specific instance by its instance_id. This is because GameMaker objects are not true “objects” in the OOP sense of the word. An object in GameMaker is not a class in Java.

This has given rise to a perception that GameMaker objects are “bad” because they mislead first-time programmers into misunderstanding Java or C# when they eventually move on to learning it. This is a strange notion — that one should learn good Java habits by learning GameMaker. GameMaker is a good language for new programmers not because it teaches you how to be better at Java, or Visual Basic, or any other language, but because it is small, simple, and easy to use. By learning GML, you learn a bit about how to program. GML need not be an ideal or pure language — it just needs to be easy, accessible, expedient, and internally consistent. Learning other programming paradigms and “purity” can come later.

Referencing an instance by the object_id serves the purpose of being easy, accessible, and easy to use. A very inexperienced programmer can understand the object_id and use it long before they start thinking about instances. And using object_id works as long as there is one and only one instance of that object.

If referencing the object_id doesn’t work consistently across platforms, that is a bug. It is a long standing and acceptable practice, and does not suddenly become “bad” because YYG doesn’t want to do the hard work of making different build targets all behave identically when running the same GML.

Using a with statement isn’t a silver bullet fix, either. For example, in the code example from the tech blog article, if we reverse the statement:

enemy.x = obj_Player.x;

we can’t “with” that away — enemy.x = with(obj_Player.x); makes no senses; with does not return a value.

What we really need to do is get the instance id of the obj_Player instance, and use id.x instead of obj_Player.x.

But obj_Player.x is more intuitive for a beginner, non-programmer. It’s the name they use for the object type is something that they typed into the game, that is visible to them in the IDE, and that they know about. An instance’s id is generated at runtime by the program, and is all but invisible to the programmer, until they learn about it.

When I first started learning GameMaker, before I’d moved from Drag/Drop to GML coding, it was months before I even knew that instances had id, or how to find out the id of an instance. Learning that was a breakthrough for me, and enabled me to do things I could not do before. But before I got to the point where I ever needed it, I’d happily made a number of projects that never once needed it, and worked just fine with object_id.variable references. I must have been a terrible programmer. Yet I managed somehow to build several projects this way. I learned enough and gained confidence, and eventually enough experience to learn the “right” way to do it. If I’d never had the easy, but “wrong” way of doing it, I might never have gotten anywhere, and probably would have given up trying to learn because it was too hard. But because the “wrong” way gave me a way that I could understand, I learned enough that I was able to figure out the better way, and the process of learning that made me a much better programmer than I would be if I had simply been handed the right way to start off, and never had to think about the problems that arise from doing it the wrong way, had the opportunity to learn how to solve that problem, and understand why doing it the right way makes the right way better.

To remain a valuable tool for beginners, GameMaker needs to keep these “bad” but easy features intact so that beginners have a foothold on getting anything at all to work.

Finally, the tech blog’s advice that “[valid, compiling GML code] behaves inconsistently on different platforms, so don’t do it that way” is simply bad advice, a band aid solution to the real problem. I think it amounts to an inability on YoYoGames’ part to deliver a product that will compile software that will behave identically on all platforms. Granted, that is not an easy thing. But because YYG has not been able to get consistent behavior out of compiled code across all platforms, they tell the user to avoid using such code rather than fix their problem. What they really need to do is make the code run consistently when compiled to all platforms. It’s not a true “cross platform” tool if you can’t do that. It might be a tool capable of targeting multiple platforms, but that does not make it cross-platform. With platform inconsistency, the result is that the developer must choose which platform to target for a project, develop for that platform only, and maybe think of porting to other platforms if it’s not too much trouble and if the extra work will be worthwhile. Or, constrain themselves to using only those features which do behave consistently across all platforms, limiting what can really be done with the tool.

If the same GML behaves differently on one platform vs. another today, it’s fine to warn about that, for now, but it should be on a list to fix that problem so that the same GML will behave identically regardless of build target, not resolve the problem by removing the feature from GML entirely.

Now, a bit of philosophy…

Breaking legacy code is generally VERY bad and should be avoided unless absolutely necessary. Whatever gains we might see from this, it doesn’t qualify.

I must question the wisdom of changing something as fundamental as variable scoping rules after so many years. GML has been around since 1999. Granted, this isn’t the first time it has changed. But programming languages need to be consistent and stable in order to remain useful.

I’ve only been using GameMaker since 2010, starting with v8.0, and so I am only dimly aware of similar changes that may have been made in years past; I generally understand that there were some significant changes between GameMaker 6 and 7, and that since the release of v7 the language has been reasonably stable. I consider myself to be a reasonably good high-level programmer, and have studied computer science at the university level a bit, but I am not a computer scientist. I have never designed a programming language, or a parser, or built my own compiler. I say this because I do not wish to be taken as having more authority than I do on these subjects. But for me to be able to do the type of programming that I do, I do depend an awful lot on people who do write compilers, parsers, and design languages and frameworks. So when they do things and it affects me, I take notice of what they’re doing.

With any programming tool, code stability should be a very important factor to consider when making changes to the tool. It is incredibly disruptive to developers who are using a language to change rules like this, breaking existing code, and especially changing the way they think about coding.

Whether the original design of GML was “optimal” or ideologically “pure”, or not — at this point the scoping rules are entrenched and should not be changed.

In my opinion, if YoYoGames wants to change the fundamentals this drastically, then they need to commit to preserving a “legacy mode” to allow old code to run with old scope rules. Call the new scoping rules GML 2.0 and provide a means of compiling the game to GML 1.0 spec or 2.0 spec, like we have many ANSI standard specifications of C that have come out over the years.

If they are not willing to do that, as I strongly suspect they will not, then the next best thing would be to provide a conversion tool that will rewrite an old project to follow the new scoping rules without changing the way the program behaves at runtime. To be useful the conversion tool must output a project that preserves the original code, creating a new project without destroying the old one, that works the same as the original project in every detail, and is built out of readable code. This is a tall order, and so I do not expect that YYG will do this either.

Yet, if they don’t do this, it’s a slap in the face of longtime users who have codebases that they have built up over years which are soon to become useless, not to mention the years of learning and ingrained habit that will now need to be un-learned, resulting in aggravation and pain as programming techniques that worked for years and still look correct no longer compile, or run with odd, unexpected behavior.

Don’t break over a decade accumulated of legacy source code by changing the language and leaving no recourse to the developers who need to support old code. Old code that has shipped and could be earning money, that needs to be patched, and therefore recompiled.

I am not against change. I welcome positive change. This is not positive change.

I’ll take some pains to make it clear that I’m not a luddite when it comes to change. I’m generally in favor of most of the changes YYG has made with GML. I don’t mind dropping obsolete functions that no one uses, or which were Windows-centric and could not work on all platforms. No one misses the CD-ROM functions.

I do miss the room transitions, and the capability of execute_string() to do interesting things like write a GML program that writes GML programs, or that can take arbitrary GML as an input and execute it, or that can modify itself. And I get bent out of shape when a project I’ve been developing or a technique I’ve come to rely on in many projects breaks because some function has been removed from the language, or because it now behaves differently.

I absolutely endorse improving GML runtime performance, but GML should not be changed to make it more like other languages — particularly when it changes the behavior of code at runtime. Performance should not come at the expense of correctness. Make it right, first — then, make it faster.

That’s not to say that I oppose any and all change.

I was happy when YYG added the ++ and — operators to the language. That change didn’t break any code I’d written before, but it did make it possible for me to code in a more C-like way that I’m accustomed to from using other languages.

I was happy when they improved the implementation of arrays. The changes made arrays more useful, and any changes in array syntax were minor enough that they didn’t disrupt me or change the way I have to think about programming with them.

I didn’t mind the change from running as an interpreted language to running as a compiled language. The performance gains here are obvious and well worth it, and the impact on the language was minimal.

The key is that improvements should be as invisible to the developer as possible. Performance improvements should mean things get faster, not that behavior of existing code changes. Changing the underlying language or framework in ways that change behavior is a sign that the implementation isn’t using abstraction enough.

“Bad practice is bad, so we’re removing it” is a bad argument.

The tech blog refers to practices such as globalvar and referencing an object name rather than an instance id to reference variables belonging to a singleton instance of a GameMaker object as “bad”. This is something that programmers who come to GML from a background in some other language are prone to saying, but it’s prejudicial. Let me, as a programmer, decide what is a good practice for me. I’ll listen to any reasonable argument for or against coding a particular way, but in the end it’s my project, and I’m going to code it the way that makes the most sense to me. When you go and take something away that used to be valid, even if you think that it’s a bad practice, it’s still harmful to me.

I will always draw the line at changing the language in fundamental ways that forces the developer to discard years of experience and have to re-learn how to use the language and re-write perfectly good code. I don’t need every last bit of performance that can be squeezed out of the metal to make a good game in GameMaker. And if I did need every last bit of performance, I’d be writing in C++, C, or Assembly, not GameMaker.

What I need most from GameMaker is for the correct code that I wrote six months ago, or three years ago, or longer, to still compile and behave the same way today, other than true bug fixes in the underlying tool.

GML should be GML, not wannabe [insert favorite language]

GameMaker was never intended as a tool for “hard core” programmers who needed the absolute fastest performance out of the compiler and are such good programmers that they never use global scope in their programs. It was intended to be a simple, small, easy to learn language that was easy to use, forgiving of small errors (such as forgetting a semi-colon) and misconceptions (such as conflating object and instance), and powerful enough to allow a person (even a professional, but especially the non-programmer) to make games without having to be a highly skilled at programming.

GML historically has been a fine learning tool, able to be learned by elementary school aged children, and has served as a stepping stone to those aspiring to learn other languages. But just because some people learn GML and then move on to other languages does not mean that GML is somehow a “bad” language and needs to become more like C++ in order to retain users who would ultimately be better served by using real C++.

GML is quirky compared to most other programming languages, in a number of ways. There are only two data types: strings and numbers. All numbers are floating point. True = 1. False = 0. Boolean expressions return true if they resolve to a value >= 0.5, and false if they’re < 0.5. Strings are 1-indexed, not 0-indexed (yet, arrays are 0-indexed). Arrays are funky. Iterators are lacking. An instance may be referenced through its object id, which works fine so long as you only have a single instance of the object. These quirks make GML different, and there are a few things that feel like mistakes, such as variable declaration by assignment, but many of its quirks are not flaws. Many seem to hold the opinion that they are, but in truth they are differences which make GML unique and it should be appreciated for its uniqueness. The value of future changes should be weighed against the value of preserving the language. Never needing to decide whether to use an int16, int32, int64, decimal, float, double, etc. when you need a numeric data type is a strength. I’m sure it must give rise to some kind of tradeoff, but who cares? I like that it’s simple, and all numbers are just numbers.

What is a bad practice in one language, is not necessarily bad in another. For example, Objects in OOP have state. OOP programmers spend a lot of time maintaining state, checking it, verifying it, getting burned by it, etc. Functional programming eschews state. They just get things done differently, but in a way that is less natural to think about for most people. Which is better? Should we get rid of Java because Clojure is the way “better” programmers write code? Should Java “fix” itself by becoming more Clojure-like? Of course not.

GML demonstrates that it is fine as it is, by the thousands of great games that have been built with it — and not some other tool. Very good games have been built using GameMaker by developers who are not very skilled with programming. This is a strength.

GML should not attempt to be other languages, and should not apologize for not being other languages. It should be the best GML that it can be — this means embracing the unique features that make it valuable, and adding new and needed features while not breaking existing code.

If you want to use a language that has different features than GML, use a different language. GML should refine itself and improve itself, not reinvent itself as something that already exists that “real programmers” purportedly like better.

GML isn’t a bad language — it’s different. The idea that there is one ideal way for a programming language to be is what’s bad. GML shouldn’t be ashamed of what it is and feel like it needs to conform to a “real programmer’s” language like C or C++.

Even if GML now compiles down to native binary code, and changing the language makes it easier for YYG to continue to develop the GameMaker product, YYG should realize that what they still need to do most is provide a stable platform for game developers to produce good games more easily. Changing the fundamentals of the underlying platform is harmful and disruptive, and should only be undertaken when absolutely necessary.

YYG should recognize that, and then strive to avoid harm and disruption as much as possible.

9 Comments

Add a Comment
  1. You make a great case, I didn’t know about proposed changes to scope and object/instance reference – and it is ominous. Making an existing system that many rely on should be avoided at all costs surely.

      

    1. Thanks; I really hope that YYG reconsiders some of these changes, and opts for greater stability/backward compatibility.

        

  2. OK I am going to bite here as you have misconstrued many things here…

    local variables are not shared between scripts only between Actions (which are code fragments not individual scripts) this is no more confusing than C where local variables are shared across a function. So your statements above are incorrect and you have not read (or understood what I meant).

    Our warnings over globalvar are simply that warnings, I personally do not like it and as a construct it will not be in the next version of GML, though that does not mean that global variables are bad just the globalvar construct in the language. In many discussions that I have seen, I see the same confusion I am not saying global variables are bad, just that globalvar is bad, they are 2 separate issues.

    The object syntax discussed above has also been badly understood as it does not work as many people think, in the second example where it is reversed is exactly the problem if the objPlayer.x is used as an rvalue (the right hand side of an equality statement) then in the past GM has effectively randomly chosen a single instance of objPlayer (it could have lots of instances) and returns the x value. The object syntax is bad because it hides what is actually happening and the work that happens behind the scenes – we are advising our users that it is a bad construct and should not be used, this is my personal opinion. We also have issues with it on JavaScript and YYC but they are implementation issues.

    As to your philosophy questions we do strive for compatibility, in fact we have gone to great lengths to ensure the GM:S is as compatible as we can and still make it a great product, but we had to make some compromises, you may not agree with those decisions and that is your prerogative, but each one was made with a central goal, to make GM:S the best cross platform dev tool that we could and to increase its reach and capabilities. We feel that we have made the best decisions that we could to achieve that with limited compromises.

    Russell

      

    1. Thanks for taking the time to respond, Russell. It’s good to have first-party information.

      I did re-read the Tech Blog article a few times to try to understand it as I was writing this blog post, and I realized that it’s possible that the way I was reading it wasn’t accurate to fact. If I could make some follow up questions to clarify my understanding, it’d be helpful.

      1) Regarding local variables in Actions, is the Execute Script action or Execute GML action considered an Action for purposes of determining scope? If I add two Execute Script actions to the same Event, would I need to worry about local vars declared in the scripts? What about if I took the identical code in those scripts and used them in two separate Execute GML actions in the Event instead?

      2) Regarding globalvar, are you saying that global variables will continue to be a part of GameMaker:Studio, but that the globalvar keyword and method of declaring global variables is going away? Will we still be able to declare and use global variables by using global.varname?

      If so, this is a much less drastic change and I can live with it, although I really liked using globalvar to eliminate the need for me to have to type “global.” every time I wanted to access the global variable.

      I can see that globalvar foo masking instance.foo can be a problem within the scope of foo if you don’t call self.foo or id.foo, but it seems to me that those are viable solutions to the masking problem, and don’t break old code that uses globalvar foo declarations. In situations where globalvar foo and instance.foo both need to exist, then you need to access the instance foo variable through the dot operator, even when within the scope of the instance. I’ve never seen this as a problem; merely trading off whether you want to type global.varname all the time, or id.varname. In reality I’d probably just write globalvar gfoo as my global foo, and foo in my object.

      3) Re object.var vs. instance.var, if it’s simply a recommendation not to use the object.var syntax, but it will continue to be allowed and work as-is, then again, I’m not at all bothered by this. When I have used this syntax, I do so knowing that I am treating the object as a singleton, and will never have more than one instance of object instantiated, ever, at one time while the game is running. It’s convenient to do this when I am placing the singleton object in the room using the room editor. If I’m instantiating it using instance_create() I can just use the return value from instance_create() and assign the id of my singleton to a global variable (if they’re still supported) and reference it that way, rather than by the object index. This is a lot less straightforward way to do it from the perspective new users/nonprogrammers (when I was a beginner I would have never understood this) — I agree it’s a “bad” practice but it’s cool that it works (as long as you only have one instance of the object you’re referencing) and makes it easier to make games when you don’t yet know how about id’s or how to get the id of an object that you’ve placed in the room using the room editor.

        

  3. 1) Scripts are self contained, it does not matter where they are called from they have there own stack frame, local variables do not leak across boundaries. You do not need to be concerned how they are called at all, the only issue is within a single event with multiple code blocks those code blocks are grouped together into a single function, so the code blocks are not discrete.

    2) we will continue to have global variables but they will need to be declared in a single location not spread out over the codebase, this makes there use more consistent. My main issue is that globablvar’s within an extension affect the main code as there are no namespaces to hide them in – so the code in the game could be written, an extension imported and used for a minor thing and then suddenly the game misbehaves because something that was an instance variable suddenly becomes global and shared amongst all instances, that is my main problem with them.

    3) I would be happier enforcing the use of that construct as a singleton (which is effectively what Javascript and YYC do, we allow it but it only affects the first instance) but that would add even more overhead, I may yet enforce that as we allow the ability to switch off runtime checking (to get even more speed out of your code).

    Russell

      

    1. Thanks for answering my questions — it does seem like the concerns I had originally were overblown. So, I’m relieved to hear this.

      With respect to #2) for extension building I’ve always had the concern about possible variable name collisions, since there aren’t proper namespaces in GML that can aid with scoping. The solution I found that works the best is to simply use a naming convention for all variables declared in the extension, which makes collisions less likely. Prefixing each variable with the extension name works well, and (I assume) is a best practice given the way things currently work. Naming a global variable declared in an extension “global.myext_foo” is easy enough to do, and works around the issue pretty reliably.

        

  4. This is great info and reassuring, thanks for responding Russel

      

  5. I understand that the limitation on globalvar can be gotten around, unfortunately not everyone is as enlightened as you, and it is something I would rather that you did not need to be so enlightened before they are safe.

    Russell

      

    1. Lol, I don’t know that it requires enlightenment… clear documentation in the manual should suffice, I’d think. It’s not that tricky to understand it, and if you simply mention it in the helpfile article on globalvar and demonstrate the code that works around the issue, I’d think that’d be more than adequate. It’s not the most obvious thing in the world, so documenting how to do it properly would be the right way to handle, IMO.

        

Leave a Reply