Map/match analyzer project

[Wormbo]

So I took a bot match log file from ONS-ColdFusion-T32 and the map's exported radar texture and ran them through the latest version of the heatmap tool.
The result were these 56 images. (JPEGs stored in a ZIP file, about 5.1MiB)
No manual editing this time, these came straight from the tool.

[Pegasus]

Pedestrian/vec/all position, damage at source/target, movement tracing, per-vehicle filtering and per-team/mixed/colour-coded visualizations; pretty sweet array of selections this tool is sporting there, got this data junky salivating . I wonder, based on the data it collects, it should be able to also do per-weapon/damtype damage/death mapping, right? Like, just the snipers/LGs or just AVRiLs, etc.. Also, point heatmaps aside, could a more vectorial representation of said damage/death origin/target pairs be feasible too, so we could study, say, GunShop/StarReach sniping habits in a more visually immediate way? Sorry to spam you with all these Qs, but this got me pretty excited. Already looking forward to when it's ready to be taken for a trial run here .

Edit: Almost forgot; are the different resolutions between the damage/death and movement sets owed to some inherent constraint or was that just a result of doing 'em in different days and just experimenting with different sizes? Tbh, going ahead towards a post-beta version of this, it'd definitely help if the raw pics were of 1024x1024 size to facilitate clearer inspection of maps' specific areas, even if the heightmaps themselves are 512s. Alternatively, users being given the option to have the heatmaps (or traces/linear segments) data on one layer and a [changeable/scalable] heightmap pic or some other, manually fed image on another could also be a powerful feature. But one step at a time, of course.

Once again, kudos to ya for resuming work on this promising tool, Wormbo.

[Wormbo]

Yes, using weapon information (weapon/vehicle used to kill/damage, but also where did players equip which weapon) and doing per-player maps is possible from the data. I could also create vehicle maps for "all flyers", "all hovercrafts" and "all/non-hovering ground vehicles". For the vectorial representation of damage I played a bit with SVG images, actually. Of course I could also try drawing these to raster images.

The mutator is ready for that test run, with the implications I mentioned before. (not white-listed, potentially fills disk space with logs, etc.) Image generation is separate from that, the mutator only writes log files with the data.

The different sizes were actually intentional, but I suppose they don't make too much sense. All images in that archive were created in a single run of the then-current state of the image generator tool. Background images are scaled accordingly, radar maps are usually only 256x256. And there's a second mutator that will help you create independent background images for perspective projections. Yes, these require map modifications and post-processing for results. I know that because it's a standard-ish workflow for JB200x maps.

In other news - I finally managed to get those pesky perspective projections calculated correctly:

(standard panorama map image and projection for JB-Aswan-v2)

[Cat1981England]

The mutator is ready for that test run, with the implications I mentioned before. (not white-listed, potentially fills disk space with logs, etc.)

Same link as before? If so i'll put it on tonight. If we leave it on for a day we should be able to see if the hdd is a bottleneck and how big the log output is.

[Wormbo]

I made some minor improvements, including the additional mutator I mentioned. The link is still valid, but I only uploaded the latest version after reading your post.

Thanks for giving it a try here.

[Pegasus]

Good to hear there's additional behavioural aspects that can be derived and visualized from the collected data. Studying, say, players' flying habits would be an obvious example of that usefulness that immediately pops to mind.

[...]For the vectorial representation of damage I played a bit with SVG images, actually. Of course I could also try drawing these to raster images.[...]

I'm curious, was the aim of that to reduce assets' size for the executable, or just to mitigate aliasing artifacts? Oh, and speaking of the visualizing executable, a troubling thought entered my mind the other day that I'd welcome any reassurance you can offer on: here's hoping it won't require any specialized environment or framework to be installed - your JREs, .NETs, etc. - in order to run, whenever the program is ready for public testing.

Regarding viewing angles and the most representative manner of portraying a map's geometry for heatmap overlaying purposes, I'm starting to wonder whether the whole perspective angle approach might be a fundamentally flawed idea since, even from a very distant top-down position (presumably the most helpful angle for the majority of cases), points near the periphery would likely still be visualized on the outer walls instead of more properly on the floor. However, since, I believe, both the game as well as UEd are inherently unable to render an orthographic projection of a map's geometry beyond wireframe mode (i.e., textured, dynamically lit and in realtime preview with all the emitters active), this may be the best choice we're left with. Of course an isometric angle can also be of help when aiming to distinguish between numerous levels/layers in visualized data and the last pic you embedded seems like such a case, but it, too, is not without its caveats, as the trace lines drawn over higher/impassable geometry easily reveal. All in all, it's a matter of picking one imperfect choice over another, unless there actually is a way to force the game to do fully rendered orthographic projection. Btw, that thick white border around the map's rendered geometry in the pic seems odd. Gives me the funny suspicion that you painted the map's skybox white for contrast purposes and then used a black paintbrush to get the final result in image post processing .

Lastly, while helping this useful tool's development along by doing a single, non-Friday/weekend day trial run on the server is certainly within reason - which is why Cat set it up and it's now live on CEONSS, folks! join in and become a statistic! - when it comes to greater periods of testing (a week or more) that are the next logical step, I feel the whitelist status loss is an issue we can't rightly ignore if we're to continue acting with the server's (and the community's) best interest in mind, and that it's something we should work together to resolve. I dunno if the proposed suggestion in one of my previous posts is a sensible enough starting point or just flat out preposterous, but it's a topic I think we'll have to tackle sooner or later. What I mean is, any thoughts on this are welcome.

[...]Thanks for giving it a try here.

Thanks for the hard work putting it together .

[Wormbo]

The heat map generator tool is written in C# for .NET 4.0. Windows 8 with its pre-installed .NET 4.5 will be able to run it out of the box, while earlier Windows version will need a separate install. However, .NET 4.5 is available via Windows Update, so adding it to a system shouldn't be a problem. I don't know about Mono compatibility, never used that before.

The good thing about perspective projections for this purpose it: You are not restricted to a single projection, although creating additional background images for them is additional (one-time) work. Once you got that work done (e.g. preparing the map for taking screenshots, post-processing screenshots), you can reuse them for any new match on that same map, and potentially on all of its edits that don't make significant changes to the geometry.
Having multiple angles at the same time - and you don't need to fit the entire map into view every time (though I'll still have to fix the code so it won't draw for things that are actually behind the camera) - somewhat adds back the 3D factor that gets lost on any kind of 2D projection. Particularly for analyzing flyer usage, seeing height information is an important thing. An additional side view can do that.

That weird white outline is a result of using an in-game Jailbreak minimap texture as background. (The minimap was created from a screenshot of a modified version of the map that e.g. excluded part of the geometry.) The texture's background actually is entirely white, but the border around level geometry has an alpha fade. The heat map generator will take an empty image, draw the background on it, then overlay the generated heat map and save everything in the file format I specified. For file size reasons I told it to save as JPEG, which is why it dropped all alpha information. So essentially it left the final image black, where the background image was fully transparent. Maybe I can find a way to use the background image's actual pixel values everywhere on the output image.

[Cat1981England]

Thanks for the hard work putting it together .

+1

--------------

Here you go Wormbo. It's been on for 22 hours without issue. It was a bit quiet today, which is to be expected, but hopefully there's enough data for you to play with there. One odd thing I did notice was that Evenmatch didn't register any changes in ut2004.ini, no idea why.

The highest hd write I saw was 13kB per second on ONS-SlatedWorld-BigAl-SP2, and that incudes all the other logs as well, so I don't think it would be a problem running with a full server.

If you want any more test runs just let us know

[Wormbo]

Thanks for the logs. Not sure what's going on with EvenMatch, but maybe this is related: I didn't see any "game ended" or "game reset" (if EvenMatch kicks in) lines in any of the logs, only "game start" and sometimes "game overtime". The log files are flushed after 1 second of idleness, but maybe there's a bug in my code that prevents them from continuing afterwards.
Anyway, the logs contain enough data for visualization, so I'll be able to use them for most purposes.

[Wormbo]

Weird, other data seems to be missing as well, most notably the initial player spawning information. That means for (probably) each player's first entire life span I can't tell which team the player is on. I could try guessing the player's team, but only if the player doesn't leave before respawning for the first time. Even worse, the guess will be wrong if the player changes teams before the first respawn.

I think I may have to rewrite parts of the mutator for robustness, but still I'd like to know why it failed. Which other modes besides this mutator and EvenMatch were running on the server at that time?