PK3 read errors

[Lactozilla]

Exhibit A: lua_files.pk3
This file was generated by a tool made in C#. This archive has two files inside the Lua directory, named test1.lua and test2.lua.

Expected Behavior
The game should load the test1.lua file, and then the test2.lua file. The test1.lua file should continuously print the string "first file" into the console, and the test2.lua file should continuously print the string "second file" into the console.

Actual Behavior
Only the test2.lua file loads. The only string that is printed into the console is "second file".

Cause
The test1.lua file is simultaneously a file and the start of a folder.

Exhibit B: lua_files_winrar.pk3
This file was created by WinRAR. This archive has the same files as lua_files.pk3.

Expected Behavior
Same as lua_files.pk3.

Actual Behavior
The game refuses to start-up, and prints this message into the the log file.

ERROR: Central directory is corrupt

Exhibit C: lua_files_slade.pk3
This file is lua_files.pk3, saved by SLADE.

Expected Behavior
Same as lua_files.pk3.

Actual Behavior
The game properly loads the test1.lua file and the test2.lua file. The console will continuously print the string "first file", and then the string "second file".

 

[Nev3r]

I'm pretty sure this is because the game expects hierarchically organized files in the central directory based on folders. It'd require some extra work regarding sprites to load files arbitrarily regardless of the central directory ordering, and It'd be slower (to what extent I don't know) than the current method.
Regarding winrar, it might be because the central dir it generates is different from what the game is expecting.
I'll post again when I get the time, however, stick to SLADE for now since it is really the only editor supported, and restrain yourself from manipulating the PK3s with other tools because the central directories get shuffled.


EDIT: if you're scripting something to generate PK3s, then make it follow SLADE'S hierarchy for now:
../folder1/
../folder1/something.Lua
../folder2/
../folder2/f3/
../folder3/f3/a.txt
../folder3/b.txt

../rootfile.a
../rootfile.b
...

 

[SteelT]

It was fixed here though... https://git.magicalgirl.moe/STJr/SRB2/merge_requests/452 I didn't realize it could also affect sprites though. I never really tested for that scenario.

 

[Nev3r]

I wasn't aware of that fix; seems to fix an ugly bug (oops, sorry about that one) related to comments and extra fields; that may have been what was causing problems for WinRAR-generated archives.

Regardless, the sorted central directory still seems to be a requirement, so you'll still get odd behaviors when not using SLADE.


I apologize for not explaining this properly before: SRB2 currently loads PK3s in a slightly similar way as WADs: instead of looking for FF_START, FF_END, it looks for the first entry called, for example, "Sprites/*", and assumes it is the folder entry, and treats it as the start marker. If found, then it looks for the next entry which DOESN'T have "Sprites/*" in its name, and treats it as the end marker. The rest of entries in between are loaded almost (if not exactly, I forget) identically to WADs.
What SLADE does is to organize those entries in a tree-like structure, just as I posted earlier. This is not a necessity in PKZip's specs, but it's required in SRB2 because of the way lumps are loaded as data. What WinRAR does, for example, is to place the latest added entry to the end of the list, instead of sorting them. This isn't a problem for SLADE, WinRAR, nor any other file explorers, but since it is unsorted, SRB2's "find start/end" method fails, and leads to entries not being loaded.


It could be possible to, instead of using markers, check every single entry individually and compare their names, without assuming folder limits. This would fix the problem for most resources, however I'm unsure about sprites; as I recall, either the markers or lump/entry ordering is a necessity of some sort due to the way they're loaded (I may be wrong, it's been a long time), and if that's the case, that bit of code would be required to be rewritten too.


EDIT2: Just in case, folders are ASSUMED to exist as entries in the PKZip central directories in SRB2; I have no clue if they're a requirement, but as far as I know, they're always generated by all of the programs I've tested.