Locally downloaded files (mods), download and cache : How does it work ?

[Harest]

Hi,

As stated in the title, i'm kinda lost there. I don't understand how the game is working in terms of when does it download a file that is considered as missing.

Context : I want to introduce a friend to RPG, so with the RPG@tmrpg titlepack. I put a server online. He'll be streaming and have a relatively bad connection. So i thought it'd be a good idea to download the mods beforehand. I also did that to avoid issues since the files are hosted by someone voluntarily on a server with 100 Mbits bandwidth. Mods are ~200 Mo files, so if several players try to download at the same time, you can imagine the issue.

The problem, that i wasn't able to reproduce a second time but still :
I had the mod locally in Documents/ManiaPlanet/Skins/Stadium/Mod/v3/, that's where it goes when from the TP you download a mod from a modpage. I moved the mods from ProgramData\ManiaPlanet\Cache to somewhere else for testing purpose.
I opened ManiaPlanet, joined the server, it was on Lighthouse map, so mod RPG_IslaDeMuerta_v3.zip. And here is my question : Why did a download got triggered ? I was alone on the server, and when i checked the Cache folder i saw a file with the same size as the mod (224 733 Ko locally) with the name p2p7831.tmp. After that, the strange thing is the file disappeared instead of eventually being renamed with the SHA-384 (edit : sha-256 + 128 bits something) hash as i thought. And i also saw at this moment that a locator file was created in Documents/ManiaPlanet/Skins/Stadium/Mod/v3/ named RPG_IslaDeMuerta_v3.zip.loc.

So :

• Why having a mod locally still generated a download of this mod in Cache ? Was there a bug or something (which would explain why i wasn't able to reproduce it after (yes i closed the game in-between)) ?
• What is the role of the locator there ? Discussing it with Skyslide, he told me for mods specifically it'd be needed if you're creating maps so it's integrated in the .Gbx file. It seems that it has other purpose. Is there a check between url in the locator (if there's one) and the required mod url in the .Gbx file to avoid the download ? Instead of e.g. checking the checksum of the mod available locally with the same name and the file required ?
• Overall, i'd just like to understand the logic there. It seems very strange.

[Harest]

I had it happening again. Joined a server on map Crater [v2] which is MassEffect_v3 mod, it downloaded the mod i had locally already, in Cache as file p2pE245.tmp.When i moved to the next map, the tmp file in Cache got removed and the mod locator i had locally (RPG_MassEffect_v3.zip.loc) was apparently modified.

And again, with Purple Plot. It's RPG_DefaultMod_v3 mod. Created a p2p76EA.tmp file in Cache. This time i checked the locator content, despite being modified the url stays the same.

So yeah, i'd like to know how to fix this, before July 31 ideally. Until this is a bug. Otherwise i'll temporarily fix it myself and do an archive after getting the SHA-384 of each mod so they'll be able to put that in their Cache folder and don't have these useless downloads hopefully.
Edit : Nevermind, it's not a simple sha-384 hash of the file content, it doesn't match up. Guess i'll remove the mods locally, then try to load maps with the mods to get them in the Cache folder with the proper hash as temporary fix instead.
Edit² : ManiaPlanet is trolling me. Instead of downloading the mod in the Cache with the hash, it recreated v3 subfolder in .../Stadium/Mod with the mod inside... And is redownloading the mod again because why not. I give up on this nonsense.

Last edit : I couldn't believe my eyes. Just loaded Supar Merio still testing purpose. Mod RPG_IslaDeMuraille_v3.zip, 245 504 Ko locally. It was in Documents\ManiaPlanet\Skins\Stadium\Mod\v3 AND in Cache called 1A6A9F922123521FEBDF26D2BE9671C389E81C0DD12EDB7C75BD1AC540E8157639D7B2D737FBB8D361F0D5DA6A14562F.zip.
Guess what ? Yes, it downloaded the mod AGAIN. Under the name of p2pF1C8.tmp in Cache. Restarting the map (by that i mean just pressing the reset button, to go back to the start of the map), tmp file disappeared and updated the locator of the mod in Documents\ManiaPlanet\Skins\Stadium\Mod\v3. Still the same content as explained above.
Note : It's not sha-384 but sha-256 of the file + 128 bits long something. I'm not sure to understand from what this something comes from. Example for a file coming from this locator : http://images.mania-exchange.com/commun ... Ads_MX.dds. All files being the exact same size, 87 528 octets.

Code: Select all

00C663AB94B96441BE184B550F0C3A76953FCE774B0BE07BDE99D86425B55F13821D2A28F911F6FF90F9708337422899.dds
00C663AB94B96441BE184B550F0C3A76953FCE774B0BE07BDE99D86425B55F13497E286BD0D432F7563BBFD8F03AE7F5.dds
00C663AB94B96441BE184B550F0C3A76953FCE774B0BE07BDE99D86425B55F130ED06F03D4115027613E1B56671A19BC.dds
00C663AB94B96441BE184B550F0C3A76953FCE774B0BE07BDE99D86425B55F13E239961DB4B5F6E4540A6BC5F024315C.dds
00C663AB94B96441BE184B550F0C3A76953FCE774B0BE07BDE99D86425B55F13DFD93D324E2420705CB62216049214A2.dds

[nighthawk4571]

Hi Har3st

Context : I want to introduce a friend to RPG, so with the RPG@tmrpg titlepack. I put a server online. He'll be streaming and have a relatively bad connection. So i thought it'd be a good idea to download the mods beforehand. I also did that to avoid issues since the files are hosted by someone voluntarily on a server with 100 Mbits bandwidth. Mods are ~200 Mo files, so if several players try to download at the same time, you can imagine the issue.

The issue of downloading multiple copies of mods, custom items and images has irritated me for years, although I've never actually complained about it. Many map authors who use such items in their maps don't bother to include the source of the bigger files (mods) in their descriptions. So when you load their map, the (e.g.) Mod downloads to your Cache, as you explained but you don't usually receive a copy of the relevant .loc file.

Therefore, we have to go nuts while searching for the source location manually, as well as finding a copy of the original .loc files.
Seriously, I learned how to swear when I started building my own MP maps .... not seriously really, just kidding NADEO.

Lately, I've encountered 'v3' versions of Mods and, in the case of Purple Plot - https://tm.mania-exchange.com/tracks/127063 , my system has downloaded the Mod three times.

If your friend with the 'poor connection' can simply use the following list to (in his own time) download what he needs - one time - then maybe this will prove to be the best way of dealing with the problem. Once you have the file and its .loc file in the correct folder, this nonsense SHOULD go away. Whether he gets files from your hosting service or places like http://rpg.vinummusik.de, ManiaCDN - https://about.maniacdn.net/ or MEGA etc , either way, he still has to download stuff at least once anyway.

Latest (v3) RPG Mods: http://rpg.vinummusik.de/index.php?dir= ... &order=asc

I know I'm not being very helpful here but, as I've had the same issue as you for a very long time, all I can say is that, until we get a standard storage/ sharing system (ONE '.loc' SITE ONLY for all Envi's/ Title Packs) and a process that everybody will use, I guess we have to live with the frustration.

Cheers.

[Harest]

As you said it "SHOULD". Unfortunately, at least for me with my tests, it's not the case. And i've mods with their locators, so currently 32 files for 16 mods in the directory quoted several times above already.

I also told him to download them (and we told to everyone wanting to join in this RPG discovery evening to do so too*), but if i've the issue, i wonder if it's more of an isolated case of mine, or a generalized one.

To be fair, i don't even see how it'd not re-download stuff. I checked the Gbx headers where you can see the file paths of each external/embedded resource needed local and remote, there's no checksums. So it basically can't check if the requested file is present locally as far as i understand it.
Edit : Okay, Dommy told me where they're stored : Documents/Maniaplanet/Config/checksum.txt. I guess i'll try to see if i find something wrong. Edit² : Nothing seems wrong in there for the mods.

*Sorry it's in french, but it basically says to downlod mods by precaution either directly in-game via the mod pages, or from the link you also put where the mods are downloaded from.

[Skyslide]

hello nadeo,

please fix that, it makes no sense at all and the rpg server is out of power soon because of that, if every player needs to download the mod for each map again, even if they are cached or installed already!

they are in

Code: Select all

C:\Users\Niklas\Documents\Maniaplanet\Skins\Stadium\Mod\v3

subfolder is no problem in mod folder i hope? im not sure about that .... maybe some of you could check that, thanks

skyslide

[nighthawk4571]

Hello Sky

Yep, this is what I have also (with a change to protect my true identity of course):
C:\Users\GreatestTMPlayerEver\Documents\Maniaplanet\Skins\Stadium\Mod\v3

Last night, after playing 'Coloured Eyes' by Exeqo ( https://tm.mania-exchange.com/s/tr/119511 ), I downloaded Tourists's brilliant replay to watch in Editor. Sure enough, despite having the 'RPG_MassEffect_v3' Mod and seeing it while playing, I found the game downloading it all over again as the replay was running.

...

To Har3st

My Checksum.txt file/s are in: C:\Users\MyName\Documents\ManiaPlanet\Config\

Cheers.

[Skyslide]

also funny thing...

> install mods manual in skins/stadium/mod/
> open game
> open new map choose mod
> save map
> close game
> remove mods
> start game
> rpg tp
> load map
> no mod
> locator did not get included
> mod got saved but locator not
> both files exist wtf?
> pls fix thx

*edit

you really have to reload game and map to get locator working thx nr2
> choose mod
> save map
> no locator
> res game
> load map
> save map
> again no fk locator ffs
> res game
> load map
> works

bug or feature

[HawkGer]

Could someone at Nadeo kindly look into this or at least provide an answer to how the mod download and cache works? It would really be a shame if this couldn't be fixed.

[nighthawk4571]

HawkGer's post was a year ago so, as no response from NADEO yet, time for a bump/////?

I.e. this issue is still present. Latest example (today) 'Tarzoum' by manu19: https://tm.mania-exchange.com/s/tr/168325.
Placing copies of the mod and .loc files in both ... Skins\Stadium\Mod and ... Skins\Stadium\Mod\v3 folders makes no difference. This mod (219Mb !!! ) starts to download every time I open the map.

Admittedly, manu19 may have pointed his .loc file to some site other than http://rpg.vinummusik.de (most common source I know of for RPG mods), but shouldn't the game be first checking whether I already have the mod installed before referring to whatever .loc file I have or wherever his .loc file is pointing?

Cheers .....

[Harest]

I just want to point out the opposite issue also exists. More details:
I opened CCP servers, most maps (if not all?) use a mod themed around Winter. The mod may just not download. This issue is not restricted to these servers, i saw many streamers playing RPG maps with no mod... Let me tell you RPG maps with no mod is atrocious and you feel bad seeing that for the author of the map.

I had the case myself for both of the 2 first maps of this CCP event. When you open the map locally though, the mod will download. So i don't really know what's different in the download management from a server vs local play.
I should note both mods for the 2 first maps have a locator in http (and not https), if it ever has an ounce of importance here but i don't think so.