Option to Compress SavedVariables

I like that it’s human readable. I deleted the glimmerfish from what the zaralek.

With what I proposed, you could re-enable it in the TOC doc. 99.9% of the time you don’t need to be able to read any of it.

Also, you can take a minified Lua document and “beautify” it in an online script beautifier to make it readable. I’m not asking for minification, just for the removal of extra whitespace and filler characters.

You could change what you want in the beautified code and paste it in the saved variables to update whatever you needed. It’s still just lua.

2 Likes

Yikes. Compression would be a great option! I can think of quite a few addon writers that would love a feature like this.

1 Like

This would be a literal game changer. My load time is around 10-20 seconds per toon, per load screen. All I do in the game now is collect (thanks, Crieve :melting_face:) and my addons are necessary for what I enjoy doing in the game. Higher load times mean I can achieve less in the time I have to play the game.

This doesn’t just benefit collectors but everyone who runs addons in WoW. M+/raiders, PVPers, casual players with limited time… load times are a problem for everyone. Great change suggestion!

5 Likes

Thanks!

As an addon developer we’re fairly limited in how addons can write / load saved variables. The whole system is very clunky and is sadly reliant on using the global scope, which at this point in WoW’s lifecycle has been extremely polluted by improper localization practices over the years. Anytime we can get anything off of the global scope it makes things a whole lot faster!

For example, if you call to an API provided by Blizzard multiple times, it’s actually more efficient to cache the function locally like this:

local GetItemInfo = GetItemInfo

If we could utilize local scoped saved variables, it’d fix so many problems.

3 Likes

I mean half of this is gibberish to me as someone who has very basic knowledge of coding, but please Blizz, if this reduces loading screen times, make it happen… The ridiculously long loading screens I have actually affects my ability to do certain quests or content. And it’s not because I have a toaster for a PC. I’ve got a PCIe Gen 4 NVMe SSD, Ryzen 9 5900X, 64GB of DDR4 RAM, and an RTX 3080. There is no bottleneck in my system that should be causing loading screens upwards of 30 seconds.

For example the daily quests at the Little Scales Daycare to do Whelpling Races are LITERALLY unplayable, because I spend so long at the loading screen that once it’s finally completed, the 5 second countdown to begin the race has already finished, the race has started, the debuff for not flying through any rings quick enough has started counting down, and by the time I ACTUALLY get out of the loading screen the debuff has 1 second left, so it’s literally impossible to fly through any rings in time, at which point you “fail” the race, and get put straight back into another loading screen to return you to Valdrakken.

I have been in loading screen loops for literally 5+ minutes, where I spend 5 seconds interacting with the game then upwards of 60 seconds just sitting there staring at my screen through 2 loading screens, before getting so frustrated I disable literally all of my addons just to be able to do these quests and then turn them back on again. If the fix is something as simple as what is being suggested, there is literally no good reason not to implement it. Please, Blizzard. Do the right thing, for the people who actually want to play this game.

1 Like

Yeah, sorry. This subject is super technical so a lot of it will be kinda gibberish to most. :sweat_smile:

Those specific load times wouldn’t be affected by this. This would be /reloads and logging in to a character when addons are loaded. They aren’t loaded again when you go through a portal.

LUA files being human readable is important for transparency in the addon community. It’s important because it allows easier understanding including fixes of the addons and their data by players with a smidge of knowledge. It keeps people honest. Especially when dealing with a community that includes addons that have separate executable programs that read them and upload the contents to websites I think transparency is a critically important goal.

Given the fact players including addon authors come and go there are addons that end up permanently abandoned but which players can help limp on for years as well as addons that are simply abandoned for an expac when an addon author quits.

It’s well and good for a major addon project like ATT to suggest this, it has multiple authors, contributors, a huge discord community, it’s not going anywhere regardless of one person’s feelings about the next expac.

With the advent of SSDs as standard for running the game off of the penalty for running even a very large amount of large addons (as I do and have done for years) is mitigated significantly. Very fast RAM also being standard and fast processors mean the impact from white-space isn’t nothing but it’s also not what it once was and grows less and less important with time IMO.

For many years I ran Auctioneer and the saved variables files from that were double digit megabyte in size (my auc-stat-histogram.lua file is 14mb for instance and that’s just one module). So believe me I’ve experienced what bad load times can be when I used to run WoW off a spinning disk back in the day with big addons. Disabling my addons dramatically changed my load times back then. Now it still does to a degree but it’s a matter of shaving off 5 seconds on login which isn’t that much.

Now I do think Blizzard could do much more to optimize things and they should. They haven’t really touched the addon system other than tossing on restrictions, the occasion new variable and lots and lots of seemingly random variable name changes that come whimsically.

One additional thing on a personal note. I like keeping backups. I do this using 7zip on ultra compression. Doing so I can take my WTF folder from 600MB in size to 30MB in size. If white-space were to be removed it makes going back into my back-ups to pull some data out a lot more painful without a stand-alone tool to re-add white-space (admittedly there might be an IDE that can do that for lua but I’ve never used anything but NPP for it with highlighting) but it’s a concern which is why I’d want easy control over it.

A toc setting is not a good solution as it is not something that most users could maintain for themselves.

It would take the power out of the hands of the user and put it in the addon author’s hands. Every addon update (and many people use programs to automate updates so aren’t even in control of that and wouldn’t be aware) would overwrite such a preference if set by the user. If one runs only 6 addons sure it’s not a huge deal though there will be times after automated addon updates that the files will inevitably be made human unreadable. But for someone like me and I suspect others who runs over 200 addons (in terms of ToC files, many are modules of larger addons obviously like DBM) it becomes unmanageable.

So instead if we must do this I propose an option be added in-game which globally sets or disables this according to player preference and which the game uses when writing the data to have or remove white-space

(Though I’m predicting it now, if they add this, hold-outs like me will be so small in number I predict within two expacs the option to go back to white-space will be removed entirely as a tiny but unacceptable additional maintenance burden)

Thank goodness.

I hate minification. Malware authors love it, proprietary software that tries to obfuscate how it works loves it. People who love to take control away from the user love it. Advocates of open and free software do not like it and for good reason.

3 Likes

Well, Minification does serve its purpose. It keeps things small and sometimes (although to can be quite the opposite!) it can make code execute faster. My primary job is JavaScript-based and so I’m quite familiar with how it works and how much of a pain it is to work with.

That said, yeah, I’d be fine with a global setting provided in the UI. I just want an option to make my load times faster and to also make it faster for everyone else. I have my tool that fixes my load times, but I want the same thing (but better) for everyone else as well!

EDIT: As far as people doing malicious things with SavedVariables, even if it wasn’t trimmed of excess characters, no one would know to look at that anyways. If it was proven that an addon author was doing something sketchy, they could be dealt with on a per-addon basis. We have a TOS on CurseForge and WAGO that outlines this.

1 Like

IMO link the tool!

I think if you take 10-15 seconds to load addon data, you probably do have an excessive number of addons?

I mean, I notice ATT takes maybe a second to load after I get into the world.

It’s a C# program that I wrote that loads the Lua (using NLua) code for every *.lua file and exports all the globals found in each document back without indenting and commenting anything. (I wrote it specifically for my WTF folder, but if someone wants it, I could possibly share it, although posting a link here is likely against TOS.)

That sorta depends on how big your collection is in ATT. My account is still (after not maining Retail for 3 years) in the 60% range account completionist wide.

This probably really only affects people using addons like ATT with high levels of account completion. I’ve noticed fresh accounts load up pretty fast. (my nephew loads in lightning fast and I stare in disbelief)

“More Data” = “Longer Load Time”

1 Like

I mean… I’m tracking “some of the things, Rogue mode (Main Only)”.

Account mode is off.
Not tracking appearances, heirlooms, illusions, flight paths, recipes ( Blizz UI has that), reputations (mostly exalted), or titles (who cares?). Not tracking unobtainable stuff or wrong covenant stuff.

So I’m at 30556/35584 overall, 85.8%.

Sub-second load time.

I’m going to guess account-wide appearances are a good way to choke it.

Crieve is asking for Blizzard to rewrite the saved variables file the only code he has is removing whitespace from the current files which is a manual process.

bumping for visibility and more discusion, i have ton of alts and i like to track as much as i can so i dont have to log into them to find stuff and sometimes the load times are interesting hehe

I know, I’m interested because I have some uses for reading the variable file myself but wasn’t keen on writing my own SavedVariables file parser.

You know I didn’t think I cared about this until you phrased it this way.

Very good point.

1 Like

Not to be negative or anything but
Blizzard wants the game to be completely playable without addons.

So, with that being their stated feelings on the matter, I doubt they’ll be catering to an addon dev.