Elysian Shadows

The following is mainly intended as a question to Falco but if anyone else has opinions or suggestions regarding the topic raised I am, of course, most interested in those, too.

I decided to ask this when i tuned in a little late to Falcos livestream on Monday, just in time to hear him express a certain disdain for exceptions with a statement akin to "nobody has time for that overhead" and going on to mention they are turned off when compiling Elysian Shadows
(I might have missed the question being addressed already since i, as mentioned, tuned in quite late and couldnt stay very long due to the timezone difference and other obligations. If this is the case I apologize for the inconvenience).

I am not going to dispute this choice, which is in my view a reasonable and rational one. Exceptions, while useful in some cases, not only complicate a programs control flow significantly, but even violate one of the most important aspects of C++'s design philosophy: you should not pay for what you dont use. Whilst no other language feature(except RTTI) has, at least to my knowledge, any cost attached in instances where it is not directly utilized, exceptions do add overhead for every single function that might potentially have an exception thrown(directly or indirectly through another function called in its body). Even though the most popular ways of implementing them, often dubbed "zero-cost" exceptions, guarantee no additional code is excecuted when nothing is thrown, their name is nonetheless a lie, because, at least in remotely modern architectures, additional code does not necessarily have to be executed to have a runtime cost attached to it
(Due to interesting issues like icache misses and some optimizations being unavailable in the face of exceptions. Here are some interesting details about exception implementations for interested readers ).
Aside from most games, even llvm/clang, one of the best written open source c++ projects known to me, forbids exception use
Those are more than enough reasons to justify turning off exception handling, especially when writing time critical code for resource constrained systems.
(Even so I would still propably measure this cost on an individual basis to check if it really is significant(which it may very well not be in many cases))

I am, however, quite interested in how you are dealing with the complexities switching off exceptions globally entails, which is not quite as obvious. Despite their apparent lack of popularity, they are nonetheless an integral part of the language. A fact not only reflected in the standard library, but even in one far too frequently (ab)used language feature that normally reports errors via throwing: new.

In general, C++ has a very nice and well designed library(with a few exceptions, like the return value of binary_search) and I really like using it. However, since exceptions are a part of the language and disabling them is, as a matter of fact, a non-standard extension(offered by almost every compiler) many parts of said library rely on them for error reporting and the behaviour without them is not quite clear. gcc has a detailed explanation of how it handles support in standard library headers, which basically boils down to:

• - replace try with if(true)
  - replace catch with if(false)
  - replace throw with abort
  (- ignore exception specifications)

If this is known and taken into consideration, most problems can be worked around in some way.
Sadly I could not find appropriate documentation for libc++ or other implementations quite as easily, so even so one might reasonably assume they do something similiar, one can never be quite sure without looking at the actual implementation, which emphasises the portability problem caused
(a casual look into some libc++-headers seems to reveal that all exception related code is wrapped with #ifndef _LIBCPP_NO_EXCEPTIONS. I didnt really bother to look deep enough to know what happens to normal new statements
(it would be interesting to know if they are automatically replaced with the nothrow variantion (for gcc/libstdc++ this seems to be dependent on which library version is linked ) )).

But all is not lost. For almost every case a standard library function throws, the preconditions could be checked manually(for example std::vector::at vs std::vector::operator[]) and/or nothrow alternatives exist(for example std::stol vs directly calling std::strtol).

Those without alternatives seem to fall into two basic categories, one of which sadly quite common in containers:

• 1. Constructors
  2. Functions which might allocate memory(stuff like insert/push_back/resize/reserve/etc.)

Constructors are sometimes forced to use exceptions, because they are really the only reliable method available to report errors happening within them. Any other method will most likely fail if a constructor for a member fails and even if this is not a problem, once a constructor has run, the object is considered fully constructed and its destructor will be called later, which may yield unwanted results if the objects state is not a valid one.
Fortunately, constructors throwing for reasons other then memory allocation failures are kind of rare, even more so in the standard library and do not prove too much a problem(Off the top of my head I can think of only one, std::regex)

The second problem, memory allocation, is a more serious one. Despite the fact that there is a nothrow version of the dreaded new, which simply returns a nullptr on failure, no standard container could use it. To provide the ultimate flexibility we all know and love C++ for, containers have one additional template type, the allocator type, which allows control over dynamic allocations. To acquire memory, the container calls the allocators allocate(), to free it uses the respective deallocate() member, being blissfully unaware of all the gory details that actually happen, which are safely hidden away and fully configurable. This is a wonderful thing(I find Howard Hinnant's stack based allocator especially impressive), even more so since C++11 significantly simplified the required interface, but that very interface proves to be problematic without exceptions. According to the standard, the pointer returned by allocate must point to a valid region of memory that can be used to hold the specified number of objects of the desired type. nullptr does, of course, not satisfy this requirement and may therefore never be returned. Since this is guaranteed, containers may - and will - rely on this return value being valid and simply construct objects without further inspection. Without exceptions, there is no way to prevent them from doing so(whilst keeping the program alive).
To further complicate matters, most containers pre-allocate more memory than actually needed (in order to reduce the number of allocations needed and fullfill some amortized complexity guarantees), which is why even with knowledge of the amount of available memory it is not generally possible to check preconditions in advance.(http://stackoverflow.com/questions/4826 ... ation?lq=1)

The only real "solutions" I could perceive would be the following:

• - Either do not use the standard containers at all
  - or use custom allocators everywhere(which you propably do anyway) and accept the fact that the only reasonable action on allocation failure is to optionally log/throw some violent insult at the perpetrator and then abort everything

I could live without (unordered_)map, (unordered_)set or list if i have to(they spread the data all over the place anyway...), but I do love my vector, so the first option would be kind of a last resort. The second, however, does not strike me as much better. I can easily think of scenarios in which an out-of-memory error could be reasonably addressed by the calling code if it could somehow be reported(and a hard error might not be desired).

So, with apologies for the long prelude, I will finally state my actual question:

How do you deal with this problem in ES? Is there some way around it that eludes me and if so, would you care to share it? Am I overthinking the issue? Does anyone know how other projects banning exceptions deal with those issues?

(As always, I beg your forgiveness for the wall of text and I am most sorry if this turns out to be a stupid question)

I don't think this is a stupid question at all. Your thoughts are both well founded and completely sensible. I'd definitely be interested in reading Falco's response when he gets time to read your post.

A note on your proposed solution:

use custom allocators everywhere(which you propably do anyway) and accept the fact that the only reasonable action on allocation failure is to optionally log/throw some violent insult at the perpetrator and then abort everything

In C, there are three schools of thought on responding to failed allocation:

• Ignore it
• Call abort()
• Return an error

I would strongly suggest avoiding the first two in any library code, unless you have a damn good reason, as it will make your library utterly unusable in any environment where crashing is unacceptable. Ignoring the failure will, in most cases, lead to a segfault, but it relies on undefined behavior and can potentially cause the code to continue running beyond the expected point of failure and crash with unknown side effects. Calling abort will terminate the client (calling) process, which does not belong to your library, but the application using it. Returning an error (such as NULL, or using some status structure, or returning an enum value with an output parameter) will allow the client to decide how to respond to the failure. High-availability systems may best handle failures by freeing unused resources in other portions of the software (clearing caches, etc.) and attempting the allocation again. Crashing takes the decision out of the hands of the client and forces it to use your model, which could make the entirety of your code useless in some applications. You mentioned logging as well--if the library does not log and simply crashes, you will not have the option to log either.

Of course, if you are the client, and you decide that you would prefer to crash on failure, that's fine. Some C applications use a malloc() wrapper to keep the error checking out of their main code: Git even uses this technique to re-try allocations:

Thank you for that note, it clarifies the issue and i pretty much agree with everything you wrote.

The problem with disabled excpetions, however, is that your third option, returning an error code(which is, imho, the best one), is not really possible when using standard library containers(or in constructors for that matter). They are designed with exceptions in mind and can - and will - therefore assume that, if allocation fails, that error is reported and propagates to the call site which may try to handle it. As a result they do not check the return value for any kind of special values and cause undefined behaviour if an invalid pointer is given to them.

This was why i put solution in "", as those are not really solutions to the problem but the only methods known to me that are at least predictable.

I don't think it's practical to use the bulk of the STL without exceptions. It was designed for use with exceptions and most implementations are written with this in mind. Any flags to disable their use are hacks and will always have some issues. The EASTL specification suggests several alternative approaches in appendices 17 and 18. If existing solutions (such as EASTL) are not suitable for your application, it may simply be necessary to provide your own container implementations--which, if largely written in STL-style, can still be used with other STL features such as <algorithm>.

Okay, so now I'm confused as shit. I'm looking through all projects for the Engine/Toolkit (Win, Linux, MacOS, iOS), and they definitely all have exceptions disabled, yet they are definitely all using at very least std::vector and std::map without any issue... At least in the Toolkit-specific code, I have no reservations against using the shit out of STL and Qt's versions of STL containers... Maybe I'm just using minimal STL in the engine and am relying on Qt's STL implementation which doesn't use exceptions in the Toolkit? I'm not even using std::string in the Engine core, either, and I'm using a custom const char* comparator with std::map.

The engine itself uses STL very, very sparingly (just std::vector and std::map in a few places). I'm trying to roll my own vector-esque general-purpose container using a contiguous, statically allocated memory pool under-the-hood, and I'm trying to move everything over to using that eventually... Currently it may very well not be able to perform up-to-par with STL, as I haven't profiled it extensively yet, and I'm not so arrogant as to believe I can out-optimize std::vector<> without really trying, but it does have spiffy C++11-style iterators, it does use contiguous memory pools, and it can easily emulate linked-list-style containers with better cache coherency, so functionally it's all there.

My main concern with STL, specifically std::map<> is on the Dreamcast. I don't even use std::strings it the codebase, and for the same reason Lua performance is not particularly great using the standard allocator on the Dreamcast, I'm worried that a lot of small heap allocations happening frequently behind-the-scenes will fragment what limited RAM I have, so I'm afraid of any non-contiguous, fully dynamic data structure in code that can run on the Dreamcast.

How do I check for errors upon allocation? The short answer: I don't. This may seem reested to you all, which is understandable, but I personally don't think it's worth the run-time cost. The engine is very, very error-friendly. You can load a level with absolutely nothing and basically play through ES as a bunch of untextured quads if all loading fails, because that's at least easily recoverable... But if I were to literally run out of heap memory in the middle of an allocation, what the fuck are REALLY my options? is the software really going to be able to continue running? For me, not without a gigantic amount of complexity surrounding every allocation introducing a shitton of overhead for something that isn't supposed to happen in the first place... The program shitting its pants sounds like a perfectly reasonable reaction to running out of heap memory to me... All I would be doing if I checked for that at run-time would be asserting and printing some error message anyway.

At least in debug mode, libstdc++ seems to be already giving me a pretty little error for most memory exceptions (I guess maybe the dynamic library is still using and handling exceptions behind-the-scenes even if my code has them disabled?), and even if it doesn't, and I continue to execute the program, dereferencing a nullptr, I have very detailed OS-level signal handlers that generate detailed stack-backtraces which would make it pretty clear to me even without the exact exception that I ran out of memory or that malloc/new shit its pants...

I just think in an engine where speed and small run-time are so critical, the performance hit for checking every container allocation is not worth the benefit, considering no matter what the program is going to be fucked if it runs out of heap space... I didn't even realize the new STL relied on them so heavily.

But I think the biggest question here is how the fuck I am able to have exceptions disabled with no problem in my projects...

Thank you very much for the answer, it is much appreciated.

Okay, so now I'm confused as shit.

Sorry about that, confusing you was not my intention.

I'm looking through all projects for the Engine/Toolkit (Win, Linux, MacOS, iOS), and they definitely all have exceptions disabled, yet they are definitely all using at very least std::vector and std::map without any issue... At least in the Toolkit-specific code, I have no reservations against using the shit out of STL and Qt's versions of STL containers... Maybe I'm just using minimal STL in the engine and am relying on Qt's STL implementation which doesn't use exceptions in the Toolkit? I'm not even using std::string in the Engine core, either, and I'm using a custom const char* comparator with std::map.

According to this page, QT does not use exceptions, except in case of memory allocation errors, in which case even QT will throw std::bad_alloc. So sadly, using QTs containers wont completly get one out of this mess either.

The engine itself uses STL very, very sparingly (just std::vector and std::map in a few places). I'm trying to roll my own vector-esque general-purpose container using a contiguous, statically allocated memory pool under-the-hood, and I'm trying to move everything over to using that eventually... Currently it may very well not be able to perform up-to-par with STL, as I haven't profiled it extensively yet, and I'm not so arrogant as to believe I can out-optimize std::vector<> without really trying, but it does have spiffy C++11-style iterators, it does use contiguous memory pools, and it can easily emulate linked-list-style containers with better cache coherency, so functionally it's all there.

This does sound incredibly interesting. Would you mind elaborating on this a little? How exactly does your container differ from std::vector<> with something like the above-mentioned stack based allocator? Is it capable of other things that are not easily possible with std::vector<> and specially designed allocators?

(I would absolutely love to see code, but guess this is a little too much to ask?)

My main concern with STL, specifically std::map<> is on the Dreamcast. I don't even use std::strings it the codebase, and for the same reason Lua performance is not particularly great using the standard allocator on the Dreamcast, I'm worried that a lot of small heap allocations happening frequently behind-the-scenes will fragment what limited RAM I have, so I'm afraid of any non-contiguous, fully dynamic data structure in code that can run on the Dreamcast.

That is kind of a different issue, but a problem all too familiar to me. Not to mention all the pointer chasing required to traverse a std::map<>. Which is why the only standard container used in our engine is std::vector<> and all uses of std::map<> were replaced with something similiar to boost::flat_map<>
(why does a reasonably simple template like this require more than 1200 freaking files in boost(at least according to bcp)?? They have some really nice stuff, but their interdependencies are nothing short of horrible)

How do I check for errors upon allocation? The short answer: I don't. This may seem reested to you all, which is understandable, but I personally don't think it's worth the run-time cost. The engine is very, very error-friendly. You can load a level with absolutely nothing and basically play through ES as a bunch of untextured quads if all loading fails, because that's at least easily recoverable... But if I were to literally run out of heap memory in the middle of an allocation, what the fuck are REALLY my options? is the software really going to be able to continue running? For me, not without a gigantic amount of complexity surrounding every allocation introducing a shitton of overhead for something that isn't supposed to happen in the first place... The program shitting its pants sounds like a perfectly reasonable reaction to running out of heap memory to me... All I would be doing if I checked for that at run-time would be asserting and printing some error message anyway.

Well, to name one possible example of what might be an option in your engine: trigger Luas garbage collector, insult the scripts author and try again. Especially once you released your toolkit to a large audience who are not as aware of the memory constraints as you and your team are, I could imagine some users (incorrectly) blaming your engine for memory related issues not reported to them. I dont know how feasible this is and it may very well not be. Nonetheless I think there are cases where useful error handling could be possible.

I agree completly that if no meaningful recovery is possible or feasible, crashing is, as you write, a perfectly reasonable reaction.

At least in debug mode, libstdc++ seems to be already giving me a pretty little error for most memory exceptions (I guess maybe the dynamic library is still using and handling exceptions behind-the-scenes even if my code has them disabled?), and even if it doesn't, and I continue to execute the program, dereferencing a nullptr, I have very detailed OS-level signal handlers that generate detailed stack-backtraces which would make it pretty clear to me even without the exact exception that I ran out of memory or that malloc/new shit its pants...

I just think in an engine where speed and small run-time are so critical, the performance hit for checking every container allocation is not worth the benefit, considering no matter what the program is going to be fucked if it runs out of heap space... I didn't even realize the new STL relied on them so heavily.

I believe your assumption is correct, if you didnt recompile libstdc++ itself without exceptions it still throws them internally. The problem I see(and which may just be a result of overthinking the issue) is that, whilst meaningful error messages and predictable crashes are possible, they are not guaranteed to happen. Dereferencing a nullptr is undefined behaviour, so with a little bad luck your engine might keep running and display unwanted behaviour on other systems.

I think this is not really new to the current standard library. As far as I know, new threw since the first standard and there never was any standardized way to globally disable exceptions
(There might be one in the future so. I think the rather new SG14(Game Development & Low Latency) might be working on something like it)

But I think the biggest question here is how the fuck I am able to have exceptions disabled with no problem in my projects...

I assume you just dont use stuff like std::vector<>::at (for good reasons) and dont usually run out of memory. This, however, is exactly why i consider the situation so troublesome: Its not that a hard error - or any error for that matter - is guaranteed to happen and it might just work flawelessly for a long time, until, one day, it just doesnt.


I apologize for the late reply and hope none of this could be misconstrued as some form of criticism of the way you do things, it is not meant to be. I find your approach reasonable and the more I think about it the more i come to realize I am just worrying about theoretical issues that dont seem to cause too much trouble in practice.

Fillius wrote:Well, to name one possible example of what might be an option in your engine: trigger Luas garbage collector, insult the scripts author and try again. Especially once you released your toolkit to a large audience who are not as aware of the memory constraints as you and your team are, I could imagine some users (incorrectly) blaming your engine for memory related issues not reported to them. I dont know how feasible this is and it may very well not be. Nonetheless I think there are cases where useful error handling could be possible.

You chose the exact example where I fully agree with you that silently returning a nullptr and allowing the program to shit its pants is not acceptable, especially on the Dreamcast where this is not at all an unlikely scenario. In this case, yeah, I would definitely handle it differently. I already know for damn sure that I am going to be using a custom allocator for Lua that uses a statically allocated pool and is specifically optimized for returning floats... In this custom allocator, I will definitely be checking for and protecting against malloc or new returning a nullptr.

I've heard some warnings from fellow developers who have used Lua extensively on the Dreamcast that the default allocator with KOS basically rapes the system. Lua makes a shitload of tiny memory allocations for every nonlocal variable, all just the size of a lua_Number which is either a double or float... Then KOS apparently has a pretty damn big heap block size, which results in shitloads of wasted space and fragmented memory... I haven't experienced any of this myself, but based on what I've heard there's no way in hell we aren't going to be using that custom allocator for Lua.

Fillius wrote:I apologize for the late reply and hope none of this could be misconstrued as some form of criticism of the way you do things, it is not meant to be. I find your approach reasonable and the more I think about it the more i come to realize I am just worrying about theoretical issues that dont seem to cause too much trouble in practice.

Haha, it's all good dude. I know I'm a very arrogant, egotistical sack of shit, and I know a lot of people think they have to tiptoe around my ego, but honestly, you could tell me all of my code was a reesting piece of shit, and IF you could prove me wrong or show me a better way, I wouldn't be mad at all. Respect is the only logical response for someone who helps to improve your codebase and helps you evolve as a developer by pointing out your mistakes or offering an opposing point of view... Especially for you, who has corrected my ass before, there's nothing but respect.

EDIT: Now that I'm looking at it, if I really am moving away from STL containers and into my own, which all have custom allocators, I guess I really am handling the majority of allocation errors. I even have a custom global new operator on Windows that forces the allocations to (16-byte?) alignment, and even there I am checking at least for nullptr returns... I am a huge fan of log systems, personally. I fucking hate having to poll OpenGL every goddamn time I think there's even a chance of there being an error being thrown from an API function... Then you have to get the error before that state is wiped out, convert some meaningless enum (that isn't even the same enum between API revisions) to some similarly cryptic, shitty string just to see what in the actual fuck is happening... It's not even practical to keep the error checking around a lot of central rendering functions that are getting called a million times a frame...

I much more like registering some kind of logging mechanism at program initialization then allowing the library/API to log error events asynchronously. I have multiple levels of debugging (VERBOSE, WARNING, ERROR), and all errors from libGyro and up are sent to a custom handler which I specify at program initialization... This is how I can display the real-time debug terminal in ESTk and where our very detailed crash reports come from. I can apply filters to look at certain levels of log output or even filter output based on source (Engine, Toolkit, libGyro, Lua). I can also disable logging globally for performance reasons or whatever instead of having to #ifdef out error polling all over the codebase... I fucking hate that.