native XML/JSON parser
complete
GTNardy
Benchmark comparison: (9.5 MB JSON file)
Benchmark 'Lua JSON Parser' (x10) took 9964ms.
Benchmark 'Native JSON Parser' (x10) took 1057ms.
GTNardy
I'm adding a ReadJSON and ReadJSONAsync to File class, they load and parse the JSON in C++ low level and then parses to Lua, I got 10x more performance with them compared to the Lua JSON library
I'm not adding XML right now as it would require much effort which I don't think it's worth spending right now and JSON is already a great direct alternative which should suffice for now.
GTNardy
complete
Voltaism
JSON already exists in the API : https://docs.nanos.world/docs/scripting-reference/utility-classes/json
Adding XML to the API will add it on lua too because nanos has a lua library (where JSON is) so you'll have to deal with this performance.
You can PR an XML parser there : https://github.com/nanos-world/nanos-world-lua-lib
-ffs-PLASMA
Voltaism: If we would have to live with slow performance, couldn't it be offloaded to a seperate thread so the server doesn't stall for half a second when parsing a XML file. So called async loading.
T
Timmy
-ffs-PLASMA: That's a good idea, but if you're parsing such big files the best practice would be to do it only on server start and cache the result even if it's done asynchronously
-ffs-PLASMA
Timmy: Unfortunately cant do that. I have tens of thousands of XML files which need to be parsed dynamicly. Once they are parsed, I can cache them but not on server start.
Voltaism
-ffs-PLASMA: what do you want to achieve ?
-ffs-PLASMA
Voltaism: I just want to parse XML files without suspending server thread for half a second every time :3
Voltaism
-ffs-PLASMA: but why you need to parse them over and over ?
Robert
Voltaism: if you want to create a multi gamemode server you need the ability to parse XML or JSON quickly to load maps quickly. Halting either the client or server thread isn't an option in this case.
---> Mutli gamemode would still require dimensions
> only on server start and cache the result
Depends on how much memory and maps you have to load and cache. For a bigger multi gamemode server this isn't an option.
There is definitively a case for a native implementation. Initially for the database there was only a module and now its native too.