Mac vs Windows texture handling

[Flippant Sol]

This is a different question, what compression algorithms does Aleph One do under the DDS container?

[treellama]

This is a different question, what compression algorithms does Aleph One do under the DDS container?

DXT1-DXT5, but it is just passing them to the video card. It doesn't have the ability to compress textures itself, which is why the opac_type / mask-combining features in MML don't work with DDS.

[ravenshining]

DXT1-DXT5, but it is just passing them to the video card. It doesn't have the ability to compress textures itself, which is why the opac_type / mask-combining features in MML don't work with DDS.

Hm, is that possibly why I haven't been able to get premultiplied alpha to work properly? I'll have to try again with non-dds textures.

[treellama]

It supports DXT2 and DXT4 if you want premultiplied alpha DDS. IIRC you don't need to set the MML attribute if you use those.

[ravenshining]

Ah, maybe that's it then. The GIMP DDS plugin doesn't seem to support DXT2 or DXT4. I've tried setting glow_premultiply with PNGs, BMPs, DXT5-compressed DDS, uncompressed DDS; none of those work.

I guess other people use Aorta for DDSing? Oh! Which I see is something you made for A1. Won't compile, unfortunately - looks like it may not like WxWidgets 3.0? - something to sort out another day.

[The Man]

When you turn down the texture resolution all the way, do you see the textures decrease significantly in quality?

Two weeks later, I’m finally getting around to answering this. I wanted to double-check to make sure I wasn’t hallucinating, and just never got around to it until now. (Sorry about that. The last few weeks have been quite busy for me, and rarely in a good way, though things are better now.)

As far as I can tell, I wasn’t. There is no perceptible difference in quality. Here are two screenshots each from the level “We Met Once in the Garden”, two with “built-in texture size” at 1/4 and two at 1. I can’t tell a difference. Maybe I’m just a dumbass and am changing the wrong setting.

I’ll try this later with Chronicles in 1.2.1 and see if that works any differently. (1.2.1 won’t load the HD Eternal textures on Windows at all, as you may recall.)

[Pfhorrest]

Can someone with Windows please try something for me:

-Download the current build of Eternal:
http://eternal.bungie.org/development.php

-Confirm that you get out-of-memory errors with the HD Textures plugin on, but not with it off.

-Move the Walls folder out of the Eternal-Walls plugin, putting it in Eternal-Data/Shapes instead.

-Move the walls.mml file out of the Eternal-Walls plugin, putting in in Scripts instead.

-Search and replace inside walls.mml, replacing every instances of "Walls/" with "Eternal-Data/Shapes/Walls/".

-Delete the rest of the Eternal-Walls plugin.

-Confirm that it does (or doesn't) work right now.


ETA: We had LT playtest alpha 5 earlier to try to rule out the same thing I'm looking into here, but it turns out alpha 5 used a plugin for textures after all. He should have tried alpha 4 instead. Can Windows users please confirm that alpha 4 (at that same link above) works fine for them, too.

[ravenshining]

When you turn down the texture resolution all the way, do you see the textures decrease significantly in quality?

As far as I can tell, I wasn’t. There is no perceptible difference in quality. Here are two screenshots each from the level “We Met Once in the Garden”, two with “built-in texture size” at 1/4 and two at 1. I can’t tell a difference. Maybe I’m just a dumbass and am changing the wrong setting.

You need to be running the latest git of Eternal - plugins included - for the quality setting to work. I had implemented a fix for beta 9, but then Pfhorrest had done some editing of the script in parallel and so my fix didn't make the cut.

[treellama]

I guess other people use Aorta for DDSing? Oh! Which I see is something you made for A1. Won't compile, unfortunately - looks like it may not like WxWidgets 3.0? - something to sort out another day.

Aorta doesn't support pre-multiplied alpha either, in favor of simplicity.

[treellama]

As far as I can tell, I wasn’t. There is no perceptible difference in quality. Here are two screenshots each from the level “We Met Once in the Garden”, two with “built-in texture size” at 1/4 and two at 1. I can’t tell a difference. Maybe I’m just a dumbass and am changing the wrong setting.

You're not a dumbass, but you're changing the wrong setting. "Built-in" is for the shapes file textures (and probably can be removed now, that was only necessary for the first generation PowerMac G3). You want "Replacement Texture Quality"

But, from other posts it looks like the MML might not have been written correctly for it until recently anyway. Once that's fixed, you should be able to scale them up/down for the appropriate performance/quality on your video card.

[Pfhorrest]

Can someone with Windows please try something for me:

-Download the current build of Eternal:
http://eternal.bungie.org/development.php

-Confirm that you get out-of-memory errors with the HD Textures plugin on, but not with it off.

-Move the Walls folder out of the Eternal-Walls plugin, putting it in Eternal-Data/Shapes instead.

-Move the walls.mml file out of the Eternal-Walls plugin, putting in in Scripts instead.

-Search and replace inside walls.mml, replacing every instances of "Walls/" with "Eternal-Data/Shapes/Walls/".

-Delete the rest of the Eternal-Walls plugin.

-Confirm that it does (or doesn't) work right now.


ETA: We had LT playtest alpha 5 earlier to try to rule out the same thing I'm looking into here, but it turns out alpha 5 used a plugin for textures after all. He should have tried alpha 4 instead. Can Windows users please confirm that alpha 4 (at that same link above) works fine for them, too.

Addendum, further testing requested from any Windows users who are up for it:

For each alpha, can you try starting a level from each different texture set? You can try:
The Far Side of Nowhere
Deja Vu All Over Again
Dysmentria
To Sleep Perchance to Dream, and
S'pht'ia.

If some of them work and others don't (in different alphas), that really helps us narrow down where the problem is.

Each alpha introduced new textures for one more texture set. I'm pretty sure alpha 1 only introduced new textures for the set used on Far Side of Nowhere, so all the other levels in that list should be unchanged since 1.1 and hopefully still work fine even if that first level doesn't. Alpha 2 introduced new textures for the set used on Deja Vu All Over Again, but the rest should be unchanged since 1.1. Alpha 3 introduced new textures for the set used on Dysmentria, alpha 4 new textures for the set used on S'pht'ia, and alpha 5 the last set used on To Sleep Perchance To Dream.

Maybe could you also confirm that current builds, with HD textures turned on, crash on all of those levels, in every texture set? It's not just the prologue that's crashing everybody, right?

[Pfhorrest]

HelviusRufus PM'd me some details of his testing. Can anyone else on Windows confirm that the same holds true for them, or something different?

Using Windows 10 and EternalXv120b9, here's what happens on my system.
If I use AO 2018.10.6, things load up OK and seem to work well EXCEPT for… see below.

If I use AO 2015.6.2 and press or select new game (or scratch start a level), as soon as the loading screen bar reaches the right side, I'm immediately brought back to the point where I clicked the exe, and the following is written to the log:

-------------------- Tue Jan 22 20:53:37 2019
Unhandled exception: std::bad_alloc (shell.cpp:381)

NOT an out of memory error.

If I use AO 2015.6.2 with HD textures plugin disabled, it works fine.


-Move the Walls folder out of the Eternal-Walls plugin, putting it in Eternal-Data/Shapes instead.
-Move the walls.mml file out of the Eternal-Walls plugin, putting in in Scripts instead.
-Search and replace inside walls.mml, replacing every instances of "Walls/" with "Eternal-Data/Shapes/Walls/".
-Delete the rest of the Eternal-Walls plugin.
-Confirm that it does (or doesn't) work right now.

After following these instructions, it crashed with the same message to the log file.

When I use 2015.6.2 with 120a4 it locks up and I get the "not responding" message and have to close the program. No message to the log file.

When I use 2018.10.6 with 120a4 it works OK.

The "EXCEPT for" from above:
Setting the mouse sensitivity low enough so the lateral action is usable, the vertical mouse action in 2018.10.6 is so sluggish that when trying to aim I can slide the mouse 8 inches before the crosshairs will even move. I've tried editing the preferences file to change the value for the vertical, but that has no effect on the vertical mouse action. Oddly enough, the mouse seems to have better action with 120a4 than 120b9 though it's still a little sluggish in the vertical.
But this is a problem with AO, not you.

If a player doesn't have any issues with AO 2018.10.6, then it seems to work OK, it's just the earlier AO's that have trouble.

And while I'm happy playing with HD textures disabled, I feel bad that there seem to be some problems considering how much effort you've put into them.

Re your latest post, as soon as I figure out what it means, I see what I can do.

vale
helvius rufus

Thanks!

So if I understand correctly:

- the newer build of AO (I don't know them by dates, is that AO 1.3b2?) loads all versions of Eternal (or at least a4 and b9, which probably means all) just fine, even with the HD textures on?

- the older build of AO (is that 1.2?) crashes with the HD texture plugin turned on, AND with the HD texture plugin disassembled (those steps I gave), but NOT with the HD texture plugin turned off (which works fine)?

(And the crash is not an out-of-memory error, but a different one; and it's more of a hang than a crash with a4, even though that should be the same texture setup as b9 with the disassembled plugin).

That's very helpful, and it suggests that the problem is something in older Windows Aleph One builds having trouble with something in the HD textures regardless of whether they're in a plugin or not.

Oh and re:

Re your latest post, as soon as I figure out what it means, I see what I can do.

Basically I want to see if the crashes are caused by any particular texture set, or equally by all of them. So if you could try the older build of AO that crashes, with the HD textures on, but try level-selecting a couple different levels besides just the first one (which uses set 4):

- Deja Vu All Over Again (which uses set 3)
- Dysmentria (which uses set 5)
- To Sleep Perchance To Dream (which uses set 1)
- S'pht'ia (which uses set 2)

That will tell me whether any of the other texture sets besides the one used on the first level also cause crashes.

Thanks again!

[Lion O Cyborg]

Can someone with Windows please try something for me:

-Download the current build of Eternal:
http://eternal.bungie.org/development.php

-Confirm that you get out-of-memory errors with the HD Textures plugin on, but not with it off.

-Move the Walls folder out of the Eternal-Walls plugin, putting it in Eternal-Data/Shapes instead.

-Move the walls.mml file out of the Eternal-Walls plugin, putting in in Scripts instead.

-Search and replace inside walls.mml, replacing every instances of "Walls/" with "Eternal-Data/Shapes/Walls/".

-Delete the rest of the Eternal-Walls plugin.

-Confirm that it does (or doesn't) work right now.


ETA: We had LT playtest alpha 5 earlier to try to rule out the same thing I'm looking into here, but it turns out alpha 5 used a plugin for textures after all. He should have tried alpha 4 instead. Can Windows users please confirm that alpha 4 (at that same link above) works fine for them, too.

Hi again. I thought I'd post this here instead of the story page this time but I've followed those instructions with Beta 9 and they work. Guess I won't need other Betas right now.

[treellama]

I don't get why you're testing it with the older Aleph One builds if the new one works. Just use that

[Lion O Cyborg]

I don't get why you're testing it with the older Aleph One builds if the new one works. Just use that

No, I meant older builds of Eternal X 1.2. I'm using Aleph One 20150620, which is the latest non beta build as far as I'm aware.

[Pfhorrest]

Hi again. I thought I'd post this here instead of the story page this time but I've followed those instructions with Beta 9 and they work. Guess I won't need other Betas right now.

Thanks! So do I understand correctly (from here and our prior discussion on the MSF) that Eternal X 1.2 beta 9, running on Aleph One 1.2:

- does not work normally with the HD textures plugin turned on
- but it does work with the HD textures plugin turned off,
- and it does work with the HD textures plugin disassembled (per those instructions you followed)

That sounds like you're experiencing a different problem than HelviusRufus is experiencing then, since he (I think) says that the first two of those are true for him, but the third is false; it does not work for him with the HD textures plugin disassembled. What you're experiencing is what I expected to happen, based on Aaron's earlier reports, which suggested that the problem was something to do with the textures being in a plugin format.

[The Man]

I will attempt to test some of this other stuff above later.

I do have a question about texture sets, though. Are we sure that OpenGL doesn’t load everything from every collection used in a given level on every operating system? I’m kind of at a loss as to why “In the Shadow of Our Pale Companion” wouldn’t work in Windows 1.2.1 otherwise. It’s based on “Monument to All Your Sins” from Starlight, which very definitely does work in Windows 1.2.1. I’ve modified the textures a little bit to account for Chronicles’ different texture sets, but most of them are exactly the same textures that are used in the Starlight version of the map.

There are some differences, though, which add a modest amount of file size. I suspect the largest is the addition of bump maps to the handful of textures the from the Sewage and Jjaro sets that didn’t already have them in the Starlight release. I can’t imagine this by itself being enough to cause a crash, because I recall the vast majority of textures used in the level being from either the Water or Lava sets. We’re probably talking fifteen textures at most.

The other big difference is that I changed a couple of other textures that were just 128x128 textures in the original map to textures that actually have OpenGL versions in the Chronicles version. Of these, I’d estimate that two or three of these became 256x256 textures, one or two became 512x512 textures, and two or three became 1024x1024 textures. The majority of textures used in the map – the overwhelming majority in fact – are 512x512 textures.

So there is a bit more being loaded to memory here, but these textures by themselves don’t seem like they should possibly be enough to cause a crash. They are, however, spread across four collections. And that’s why I still have questions.

The thing is, “Pale Companion” does use a lot of textures. But I have very little doubt that Chronicles’ Pfhor levels use even more. The Pfhor collection by itself contains 208 textures, of which the majority are 512x512 or 256x256 (about fifteen are 1024x1024). All have bump maps (only a handful of Chronicles textures don’t have bump maps), and quite a large number also have glow maps. As far as I remember, every Pfhor level loads in Windows 1.2.1. I just tested “Room a Thousand Years Wide”, the largest of these by polygon count, and it worked fine.

If “Pale Companion” is only loading textures used in the level, I can’t imagine there being more than about 70 or 80 of them. I suspect a lot if not most of the Pfhor levels use more textures. If “Pale Companion” is actually loading each set in its entirety, there are probably 481 textures. If I remember correctly, the Water collection has 127; the Lava collection has 100; the Earth collection (Sewage slot from the original game) has 127; the Sewage collection (Jjaro slot) has 127. No other level in the game, as far as I can recall, loads textures from four collections. I’d be marginally surprised if any other level in the game even loaded three.

There are a few other possible root causes of the crash, though. One is the size of the level. It has 1,843 polygons (and 405 objects), and it’s possible that adding this many polygons to the number of textures used in the level is more than Windows 1.2.1 handle.

“Pale Companion” is the largest level in Chronicles by polygon count, but not by much. “Return to Yggdrasill” has 1,760 polygons, and I’d be extremely surprised if it weren’t actually the more visually complicated of the two levels, since about 1,600 of those polygons are divided between two… we’ll call them rooms, for lack of a better term, with only the trunk of the eponymous tree breaking them up. This slows down a bit on Windows 1.2.1 towards the edges, but it works fine. However, I’d also estimate that the level probably only uses about 20 textures, and it only has 57 objects.

Of the Pfhor levels in the game, the largest is probably “Room a Thousand Years Wide”, which only features 1,090 polygons, along with 460 objects. “Dimensional Bleedthrough” has 1,040 polygons and 554 objects, and “The Paradox of Tolerance” has 1,034 polygons and 569 objects. These also seem to work fine on Windows 1.2.1.

One idea I was pondering, though I haven’t gotten to it yet, was writing a Lua script to output a list of the textures used in each level to a text file (if that’s possible). That might help confirm whether my suspicions that the Pfhor levels use more textures is correct. If it is, then it’s possible their sizes might be the deciding factor rather than the number of collections used; I think “Pale Companion” probably only uses about three 1024x1024 textures, but I might be mistaken there.

In any case, if Windows 1.2.1 isn’t attempting to load all the collections used in each level in their entirety, I’m at a loss for why it has trouble with the Chronicles version and not the Starlight version. They’re not identical, but I can’t imagine Chronicles adding more than about 100 MiB of data to the level; I’d be surprised if it even came anywhere close to that. Including the crude bump maps I’ve created, the Freeverse M2 textures add up to about 2 MiB each when encoded in .dds format at 1024x1024. By themselves, the crude bump maps I made for the Infinity textures (which are 512x512) add up to about 350 KiB each. And what clinches my confusion is that disabling bump maps and bloom doesn’t affect whether the level loads.

I am, however, open to alternative suggestions as to what might be causing the issue. Apparently an “out of memory” error isn’t even always an out of memory error, though I’m still confused as to what else it might actually signify. Further suggestions for troubleshooting are welcome.

In any case, Chronicles at least does seem to run in 1.3, so people now have an alternative to simply disabling OpenGL for that level. But I would like to know why it doesn’t run in 1.2.1.

[HelviusRufus]

I don't get why you're testing it with the older Aleph One builds if the new one works. Just use that

I have to use the older version (2015.6.20) since the new one's (2018.10.6) sluggish, erratic, mouse action makes it unplayable for me but, as I said, I don't mind playing without the HD textures.
Oddly enough, though, manually changing the vertical sensitivity in the preferences file has little or no effect on the mouse action; i.e. Hor. 3723, Ver. 3723, is no different than Hor. 3723, Ver. 32695 in the new version.

It is an intriguing problem i.e. why you don't get the bad_alloc error in the new version.

Unless, however, there are huge masses o people clamoring to use the older version of AO, it's not worth slowing down the Eternal X production schedule to figure it out. It appears that most people are satisfied with the new AO.

[HelviusRufus]

- the newer build of AO (I don't know them by dates, is that AO 1.3b2?) loads all versions of Eternal (or at least a4 and b9, which probably means all) just fine, even with the HD textures on?

YES. and 2018.10.6 = 1.3b2.

- the older build of AO (is that 1.2?) crashes with the HD texture plugin turned on, AND with the HD texture plugin disassembled (those steps I gave), but NOT with the HD texture plugin turned off (which works fine)?

YES and 2015.6.20 = 1.2.1

(And the crash is not an out-of-memory error, but a different one; and it's more of a hang than a crash with a4, even though that should be the same texture setup as b9 with the disassembled plugin).

YES. the error is bad_alloc.

OK, no eureka moments. EternalXv120b9, AO 1.2.1 (2015.6.20), Eternal HD Textures plugin turned on.

Deja Vu: exits to folder after loading; bad_alloc message.

Dysmentria:

• 1. Scratch start, play for a short while then either
  
  • A) freezes up for less than a minute and
    • a) exits to folder or
      b) comes back alive again then exits to folder; no message in log; or
    B) exits to folder without warning; no message in log.
  2. Scratch start, save game, get killed, tab to respawn and exits to folder with bad_alloc message.
  3. Scratch start, save game, play for a while, save game, get killed, tab to respawn and exits to folder with bad_alloc message.
  4. Open saved game (either the first one or the later one (see 3 above)), get killed, tab to respawn and exits to folder; no message to log.
  5. Open saved game (either the first one or the later one (see 3 above)), play for a while then freezes/exits or just exits; no message to log.

To Sleep: exits to folder after loading; bad_alloc message.

S'Pht'ia: exits to folder after loading; bad_alloc message.

[Pfhorrest]

Thanks for checking that!

On the Discord tonight, LT confirmed that everything works fine in AO 1.3b2, as in it doesn't crash, but it's inexplicably unplayable slow for him, only in some circumstances, even though he has a better machine than me and it works fine for me. W'rk also tested it out and said that it seems to work mostly fine in 1.3b2, but sometimes has unexplained slowdowns even though he has a really high-powered machine that shouldn't be slowed down at all by anything Marathon should be able to throw at it. I think Aaron also said that 1.3b2 is just unplayable slow for him, not crashing?

[The Man]

That’s correct. Everything loads, but I get slowdowns that completely ruin the gameplay about once a minute. Basically the same slowdown issues LT is experiencing, from what I can tell. The music also stutters inexplicably, and this seems to be the case even when I have all the plugins disabled.

I wonder how much of this is dependent upon OS and graphics card etc. We’ve been assuming all these problems are the same problem, but if people are getting different performance and different errors when the game crashes, they might not be. I’m running Windows 7. Windows 7, Windows 8, and Windows 10 are really three completely different OSes, though, with some code in common but some completely different. What OS are LT and HR on? That might help us narrow down whether these are even the same problem.

I’ll probably look into following some of those instructions too. I’ve really been neglecting Eternal this last week or two; since I got RADIX’s new Fighter sprites I’ve basically just spent most of my free time playing Chronicles. I have fixed a few issues, but still.

[treellama]

I have to use the older version (2015.6.20) since the new one's (2018.10.6) sluggish, erratic, mouse action makes it unplayable for me but, as I said, I don't mind playing without the HD textures.

Well, I guess my point is, we aren't going to fix anything in the older version of Aleph One, and it's already fixed in the new version. I mean, if you figured out specifically what's wrong in the old version, we could fix it in the new version...but it's already fixed.

I realize the mouse is all jacked up right now in 1.3b2, but hopefully that will be rectified eventually.

[treellama]

We’ve been assuming all these problems are the same problem, but if people are getting different performance and different errors when the game crashes, they might not be.

Speak for yourself. Stuttering video frame rate, crashes, freezes, and audio stuttering are four completely separate things.

[snakefinger]

I'm on Windows 7 using AO 1.3b3 and Eternal will crash about 75% through the loading bar when starting a new game without any error messages. The only way to fix this (while keeping plugins and OpenGL enabled) is to lower the Replacement Texture Quality from Unlimited to something else, even then the game will sometimes just close out without an error message when you are about to switch levels meaning you have to lower the texture quality even more.

[The Man]

We’ve been assuming all these problems are the same problem, but if people are getting different performance and different errors when the game crashes, they might not be.

Speak for yourself. Stuttering video frame rate, crashes, freezes, and audio stuttering are four completely separate things.

I think I might’ve meant that we, the Eternal devs, had been treating all of these as though they were all a result of the Windows version’s memory management, but I should’ve clarified that a bit more, yes.

I'm on Windows 7 using AO 1.3b3 and Eternal will crash about 75% through the loading bar when starting a new game without any error messages. The only way to fix this (while keeping plugins and OpenGL enabled) is to lower the Replacement Texture Quality from Unlimited to something else, even then the game will sometimes just close out without an error message when you are about to switch levels meaning you have to lower the texture quality even more.

We haven’t fully narrowed down the causes of this, but for some reason, in my experience, beta 2 works fine with everything at “Unlimited” for a lot of levels that seem to crash beta 3. Beta 2 also hasn’t had the slowdown issues for me on Windows 10, but that might just be a product of my new computer being nine years more recent. (My old Windows 7 box will no longer even run; I will hopefully at least be able to recover the at some point.)

Treellama’s LAA build also seems to be more reliable than the standard build of beta 3. I don’t know what he changed, but only a few levels crash on this version. However, I can’t get either beta 3 build to render films; all of my films of the final release have been rendered with beta 2. (My films of the development releases were rendered using the Mac version of 1.2.1, hence the different aspect ratio.) I don’t think I had to lower the replacement texture quality for most of them, either; there may have been one or two exceptions.

You might find it helpful to follow this link to see the most recent discussion of this topic and see if any of it is relevant. Not all of it is related to the Windows crash issues, but I’d say a good 50% is.