debug get_objects_inside_radius #3723
Labels
No Label
1. kind/balancing
1. kind/breaking
1. kind/bug
1. kind/construction
1. kind/documentation
1. kind/enhancement
1. kind/griefing
1. kind/invalid
1. kind/meme
1. kind/node limit
1. kind/other
1. kind/protocol
2. prio/controversial
2. prio/critical
2. prio/elevated
2. prio/good first issue
2. prio/interesting
2. prio/low
3. source/art
3. source/client
3. source/engine
3. source/ingame
3. source/integration
3. source/lag
3. source/license
3. source/mod upstream
3. source/unknown
3. source/website
4. step/approved
4. step/at work
4. step/blocked
4. step/discussion
4. step/help wanted
4. step/needs confirmation
4. step/partially fixed
4. step/question
4. step/ready to deploy
4. step/ready to QA test
4. step/want approval
5. result/cannot reproduce
5. result/duplicate
5. result/fixed
5. result/maybe
5. result/wontfix
ugh/petz
ugh/QA main
ugh/QA NOK
ugh/QA OK
No Milestone
No project
No Assignees
6 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: your-land/bugtracker#3723
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
So far, get_objects_inside_radius is always at the top when I watch the server from
perf top
. THis is only an example, sometimes it goes up to 60%:So I wanted to know what it does (GOIR is short for get_objects_inside_radius):
For some reason its mainly the pressureplates creating that much lag:
Still, even 500 us is only 0.5 ms, there must be a lot of those around to cause that much lag. However they do not show up in the profiler.
ActiveObjectMgr
sounds more like entities than nodes to me.I know mesecons has a setting to control pressureplate check interval with 0.1 as default.
For fun I set it to 0.2 in singleplayer and the difference was barely noticable, even 0.3 wasn´t bad. Curios how noticable it would really be on a busy server
getObjectsInsideRadius
is hugely inefficient, and all calls to it will essentially have the same run time, no matter where it is executed and what the radius is or how many entities are actually inside the radius. it'sO(n)
in the total number of mobs on the server.this is because it iterates the list of all active mobs and checks whether they're within bounds. because we've got so many active mobs (2500-4500 generally) it takes a relatively large amount of time.
both mobkit and mobs_redo have mobs check their surroundings for potential targets to attack or run away from at regular intervals, so it's a relatively expensive operation that gets called a lot.
what can we do to improve this?
did some actual performance testing, my comments above might not be great...
depending on the size of the volume/radius, the amount of time it takes to construct the list can range from 100us (r=0) to 500us (r=54000), indicating that creating the list to return is actually more expensive than iterating over all entities and testing them.
also, "get_objects_in_area" and "get_objects_inside_radius" had similar upper/lower bounds
however, note that 60% of .044s (server step) corresponds to about 53
get_objects_inside_radius
calls...a rough audit of petz/mobs_redo/mobkit/water_life doesn't show me anything that looks like
get_objects_inside_radius
getting called even once per step per mob, though.For some reason this inspired https://gist.github.com/Niklp09/0bcf6851b7cb9a957711adc2ffb2993e
If constructing the list and passing it on as a parameter is a major issue, then an additional paramter to tell the engine to only search for objects that are known to actually move (mostly mobs, players, and - far less intresting - items in tubes) might be intresting.
it can be if there's actually a ton of mobs within the region, but in our case, the run-time scales more because of the # of entities on the server, than the # of entities in the area.
data structures exist for keeping track of the position of objects in 3d, both calls should be
O(log(n))
instead ofO(n)
in the # of entities on the server. the calls are taking possibly 100s of times longer than they ought to.i was curious, so i ran some tests.
the first test matches no objects (usually). the second test matches every object. with 5122 active on the server, the first test took 1/4 of the time of the second, which is a sizable overhead.
i made a new mod which uses a binary search tree to more efficiently get objects in area/radius https://github.com/fluxionary/minetest-oia
though because it's in lua and not c++, the overhead means it's not efficient until there's around 1000 active entities. it does seem like it'd provide a performance benefit for your-land though.
Let's throw it onto the testserver?
i've done some more testing and ... it's not so cut-and dried. oia under-performs if there's hundreds or more active objects within the relevant area. also, i've discovered that enabling the "override builtin functionality" feature causes multiple issues w/ other mods, for reasons i'm not quite sure about (it seems that the 3d armor stand will keep trying to re-create an object if it can't find itself immediately after creation?)
also, this sort of performance-related functionality is hard to test w/out a lot of players loading a lot of areas w/ a lot of mobs.
in one test, it sped things up by almost 100x. in another, it slowed things down by about 20x.
there's still potential here, but it needs more tweaking. also, probably, i should try to actually code it up in c++ in the engine, now that i've got some data that the approach is successful.
This is relevant:
your-land/administration#188
Or, more simple way is using world-loading-anchors...
We should count which mods use
get_objects_inside_radius
and create a script that places anchors and spawns mobs/whatever trying to simulate the YL.Also would be interesting to get log like this:
and plot it on the map, coloring points by mod for example... Could help making anchor simulation closer to reality.
Since you recreate whole tree on the fly anyway, you could dynamically switch it on and off, but:
Fixing the engine seems like the most sane way to do this. I feel like even something minimal that just uses 2d sparse grid (ignoring the Y-coord) or a sorted list should increase performance...
Exactly. Note that any kind of reasonable structure may slightly increase cost of updating object's position (i.e. possibly migrating to a different leaf in octree, possibly having to rebalance the tree occasionally.), although that overhead will not be large in average.
Currently the objects are stored in unordered map, keyed by their ID.
reducing the "nearness" issue from 3d to 2d doesn't buy us much in terms of data structure complexity. metrics on more than 1 dimension don't "sort" naturally.
i've got a couple papers to read and some experimentation, but i do think it'd be possible to create a self-balancing kd-tree, though it won't have ideal performance characteristics. the trick then becomes implementing it in c++ and getting the engine folks to use it
Could we replace get_objects_inside_radius with a cheaper tech?
All functions that aim to detect players could instead use a "detect_player_in_radius", which is probably much cheaper. Especially if we cache the player pos once and update only after each step.
We found pressure plates terribly expensive. They fire each 0.1s ? Would they stop working if we made them detect only every 0.3s ?
Yes. Especially if people sprint over them with crystal boots.
Replacing object detection with player detection would be cheap way to regain performance ... unless it breaks something. Are pressure plates supposed to react only on players, or should they trigger for mobs too?
Do kd-trees fit this use case? It seems like they are mostly used for static objects? If moving one object sometimes will trigger rebalancing of a large part of the tree and sometimes be quick, then such uneven load is not good for a realtime game (I guess this may be what you mean by "not ideal performance characteristics"?). Or are you planning to just rebuild it each timestep?
Maybe we should learn from existing game engines...
In some other games stuff like this is done via collision callbacks, and instead of querying objects_in_radius, you just have a sphere that other objects will "collide" with. In case of a pressure plate - if it does not move, and nothing moved around it, then you don't need to call any special pressure plate code at all.
Rebuild each timestep? Well, with that approach you can go back to searching whole list each time :)
kd-tree are mostly for static objects. What we'll have probably here are players scattered somewhat randomly across world (perhaps with larger density towards haven) and the active objects mostly in some clumps around each of the players and across world anchors (not sure if those are used here at all).
We could have sparse map of sectors (perhaps 80x80x80 aligned to mapchunks) and maintain list of object IDs in each of the sector (most sectors would not be in the map, if the sector gets empty, it is deleted, there would likely be roughly 8 sectors per player at a given time?). Looking at objects in vicinity of some point would likely end up looking at single sector or maybe 2/4/8 of them if near edge or corner. After moving each object, we need to check if it has not crossed the boundary (that would not happen too often in average) and if yes, move it to new sector.
For large ranges (>38) - those would probably be limited to some staff/admin commands, usual items or commands would work with smaller ranges, you can resort back to "search whole list" approach.
The plate could be optimized to skip few next callbacks if at last callback nothing applicable was near it. But once the player (or mob?) approaches its vicinity, frequent step would be necessary for reliable step-on detection.
iterating all players in lua is definitely cheaper than iterating all mobs in c++. but pressure plates aren't the biggest offenders are they?