Map/match analyzer project

[Wormbo]

As a new side project I started with a tool consisting of three main parts: a mutator to send statistical data about game state, a listener application that receives this data and stores it in a SQLite database file per source. This database can be used to generate interesting graphs and numbers from a game, in a different way than the built-in stats.

The latter part is not very advanced yet, currently I can only plot player movement during the match. The result is something like a "heat map".

This is a plot from an AS-Mothership botmatch. That cat face shape on the right side is the space fight part of the map, while the small things on the left side are the defender spawn for the space fight part (the very small thing) and the indoor part of the level.
The tip of the "cat ears" are the attacker spacefighter spawns, while the short parallel lines in the middle are the defenders' hangar exits. The vertical lines near the bottom center are attackers approaching the docking bay, apparently without much intervention by the defenders. the horizontal "whiskers" are attackers going for the upper shield generators, while the diagonal lines are attackers going for the lower shield generator.

Here's another example of a botmatch on ONS-IslandHop-HoverMeUp, the hover tank beta map:

If you know the layout of the map, you can almost guess from the plot that red (in the bottom left) lost the match. Most of the action happened between the red core and its prime node, as you can see from the brighter colors there. The blue core on the right side saw no actual fighting action at all.
Another thing this map shows, is how much bots tick to their path network. They fan out a bit when traveling between nodes that are further apart, such as the hover path nodes, but mostly they stick to the lines you'd also see in UnrealEd. If this was a match between humans on a server, the result would have looked very differently.

That's what I'm ultimately trying to do: record movement data on a server and use the result as an objective view on the layout of a map. If certain nodes on an Onslaught map aren't really used, or if a node is a constant chokepoint, then this fact will be visible on the heat map. Currently the entire thing is pretty much in a proof-of-concept state. The listener part is a .NET console application for Windows and the mutator and listener communicate via UDP. That means the game and the listener should better run on the same machine, because the protocol doesn't handle packet loss at all. And that machine would have to be a Windows machine. I doubt that kind of setup is in widespread use among Onslaught servers.

[Pegasus]

Holy crap, a heatmap analytical tool! Such visualization utilities are the holy grail of post-release information sources for game devs, amateur and professional alike, and something I've only dared dream for ONS for years. It's the kind of thing that ppl expect studios like Bungie, DICE or Treyarch to wield and, perhaps from time to time, share some crumbs of its benefits of with the rest of us mere mortals. Massive kudos to you, sir, for working towards evening that playing field for this old game's creative community!
Of course, movement is only one of the players' behavioural habits/patterns that you can examine and assess for map optimization purposes with such a tool at your disposal, with kills/deaths spatial tracking being its most significant complement, so I do very much hope that now that you got the iron hot, so to speak, you'll consider expanding the data gathering features for those aspects as well. Maybe have different axis visualization options too, so that the height dimension relevant data can also be reserved and visually represented. Just imagine how much more we could refine maps if we were able to tell - beyond anecdotal and first hand experience impressions, I mean, and with actual, statistical evidence - what parts and areas of the geometry players are getting the most use out of and which ones are left neglected because of underlying poor design issues, all that made visible at a glance. Also, being able to, say, tell what parts of a map turn out to be the contested hotspots, possibly even in contrast to previously formed impressions, due to a map's latest version's changes, or even see what type of weapons do most of the killing in particular areas (sniping and avrils vs more close quarters DmgTypes) and having all that be useful data that we can throw in the middle of a debate on a new edit proposal to better plan the course from there would be invaluable. Man, this is some juicy surprise you've sprung on us today, mister Worm .

Btw, bit of a setback about its platform requirements for now, but assuming this manages to get through a beta phase, I'm hopeful that at least a few ppl will care to help you port it for linux too - esp. since this tool seems to be gametype-agnostic, therefore it could help everyone, and there's still plenty of qualified, tech-savvy ppl around enjoying this game. Keep at it!

[Wormbo]

I can actually also generate heatmaps for pickups, spawns, deaths and where players took damage. It's all in the data being gathered. One tricky part is figuring out where the damage came from. UT doesn't track this and there's no easy way to grab this data in the game. The closest thing would be to use the attacker's location at the time the damage is taken, but that's quite inaccurate in some cases. (e.g. fire Redeemer, jump into Manta and speed off in a completely different direction - Redeemer hits, but you are already busy elsewhere, maybe even have respawned already)

[edit]
I also thought about just dumping the data to a log file from the mutator itself, so no separate listener would be required. Obviously the log file would have to be parsed then, but that's probably a one-time thing afterwards anyway. What worries me more is the amount of data the game will have to write to disk. Currently player locations are captured 10 times per second. This resulted in a 32MB SQLite database file after 1.5 hours with 16 bots on Red Planet. A 32 player server would generate the same amount of data in half the time, possibly even a bit more due to more damage being applied.

[Pegasus]

Oh, glad to hear you can already track deaths, kills and damage types. Props for forethought there .

As to the origin logging of long traveling projectiles, couldn't you just grab the necessary info at spawn time, i.e. as the instigators weapons' Fire function is executing? Besides, map-wide visualization of the question "where did hit damage from every weapon come from?" moves past the scope of 2D point data placement (where it occurred) and becomes a matter of overlaying a spaghetti of vector data (origin point -> hit point) on top of a map; for massive amounts of such info - say, in maps with contested areas offering both short-range as well as long-range exposure like GunShop's middle node - I'd imagine the end result would be a bit more, shall we say, messy. But still, I don't mean to dissuade you from trying to expand anything at this point; if a feature has conceivable value to mapmakers and can be implemented, you could certainly give it a shot and then see if the result makes sense to keep.

[Wormbo]

Detecting the fire event isn't quite as easy as you'd think. Projectiles don't trigger the usual replacement events by default and most mods with hit statistics replace all weapons to hook into the firing logic. In UT1 (and I think also in UT3) there are "owner events" for firing weapons, but that specific even isn't available in UT2004 either. Oh well, I'll figure out something that hopefully doesn't impact performance too much.
I suppose for maximum usability, the presentation tool would have to allow the user to customize the output as much as possible. For damage that would probably include specifying a minimum and maximum range, limiting weapon/vehicle types and so on. Another thing I thought about is a kind of directional overlay: what is the predominant movement direction in a certain area (I'm sure the movement map will look messier when humans are involved), optionally only for vehicles or infantry, or what is the predominant aiming direction?

The next thing I'll try to do is making the heatmap team-specific. The tricky part here is the way I stored teams. I might have to rethink the storage part anyway for platform independence.

Oh BTW: All data is stored in 3D, I just don't have a good way to present it yet.

[Pegasus]

Been googling around for "heatmap 3d data" visualizations, but haven't come across anything actually useful. Contours can't help, color specific alterations (saturation, opacity, hue) can't help when different height data densities would just add up to a mess... I just dunno what else you could do besides actually draw the 3D space itself with the coloured points all over and just offer a slider or setting for showing/hiding dots based on density :/. Also, just like with projectile direction, depicting movement direction is more complex than just presence/activity points. Are you sure it'd be worth it? I mean, if I can know how many footsteps per point per match were taken, does the vector info itself help me any more in understanding player behaviour? Wouldn't movement direction from/to any specific point be an obvious derivation between its density and one of similar value among its neighbours?
In your IslandHop heatmap, for instance, bots' strict adherence to pathnodes means we get lean and "beaten track" kind of trails, but for maps without bodies of water where players could actually be stepping on anywhere - say, an indoors map like MasterBath - you'd get multicoloured blotches around the battlefront (hotspots) just by visualizing the point data. Would vectors add anything more at that point to be worth figuring out how to draw, in 3D no less? Anyways, not a major issue, just thinking out loud here.

[aximon]

Similar to what she did: (http://www.artificial.dk/articles/alison.htm)

Makes for some interesting images.



1min 1sec von *Alison* auf Flickr


profile von *Alison* auf Flickr

[Wormbo]

Working on team-colored maps:

This is a capture of a MasterBath Renovation v2 match with 32 bots. For some reason blue didn't take their own cupboard node, and red didn't bother either. Also the bots never used the showers. The many dots around the bathtub node are bots just standing there and doing nothing. I didn't spectate the match, but it was probably one-sided anyway. As you can see, there's hardly any blue on the red side, while the blue side's bathtub node has quite a lot of red around it. Those bright dots near the blue core look suspicious. Did bots get stuck there?

It gets a bit tricky in game modes with more than two teams, but it works as well:

That's CTF4-FaceClassic with 16 bots. The gold team's color is hard to distinguish from a red/green combination, but then again hardly anyone will still work with these game modes.

[Pegasus]

Interesting to see this tool progress and its visual output for more familiar maps, even if it's still just tracking bots; quite artistic too, although using up the colour range to represent the different teams instead of data density would probably preclude the resulting image from being called a "heatmap" in any traditional sense. Eager to see one for frags (damtype agnostic) or deaths too.

[...]For some reason blue didn't take their own cupboard node, and red didn't bother either. Also the bots never used the showers.[...]

Hmm, that is a problem. Did you try soaking them?

...

...

I'll show myself out >_>...

[Pegasus]

At the risk of sounding like a bother, any progress on this? It'd be quite interesting to see this tool move sufficiently enough into beta that a version of it can be put through its paces on a live MP server like CEONSS, see what kinda data (and additional resource load) it'd generate, not to mention pretty cool too . Here's hoping...