Combat data availability.

[Nard]

About account/char encoding: If data are published yearly, with no overlapping time, each time with a single shot encoding, it will make identification rather difficult and almost impossible if you make a time transform, even if monotonic.
suggested time transform:
if t(n-1), t(n) and t(n+1) are successive server event times:

Code: Select all

tn=t(n-1)+rand *(t(n+1)-t(n-1))

(perfectible, loop is not vectorizable )
When combined to random network delays and provided that all players are not on the same map and in sight of MadCamel's client, it should be enough to put a serious mess in his correlation trials . A similar approach can be used for account numbers to be able to keep the order.

[bell chick]

just make it opt in or provide an opt out. i dont like how it could make it easier to stalk some players if someone was able to crack anonitimization.providing an opt out option would still get you most of the community with a way out for those that really dont want their info public

[o11c]

Perhaps we could do something like:

• Try to get some extensive data on the test server, and release it.
• Let people write "programs" that try to produce conclusions.
• Run the programs on the data privately.

Combat is really not my specialty ... the only thing I know is that we need to really cut the defense rating of armor, and that that will required tweaking lots of other numbers.

[Nard]

This discussion is sterile. I opt out, and incite all players to do so. Keep your data for your small club, this is the last contrib, I'll make about statistics.
Almost all people who are able to retrieve info from these data are in TMWC or satellites so theoretically trustable. So why did you make this post if you think people who are not in your small club are not trustable?

just make it opt in or provide an opt out. i dont like how it could make it easier to stalk some players if someone was able to crack anonitimization.providing an opt out option would still get you most of the community with a way out for those that really dont want their info public

Data can EASILY be treated so that the effort needed to retrieve Trustableinformation about chars and role play has nothing to do with its utility. No Real Life information can be retrieved from them or i didn't understand a thing.

• Try to get some extensive data on the test server, and release it.
• Let people write "programs" that try to produce conclusions.
• Run the programs on the data privately.

The problem is not the programs, but the data themselves. The programs already exist and are open source.

No public data means untrustable conclusions. In statistics open source means open data.

Nard

[Nard]

i dont like how it could make it easier to stalk some players if someone was able to crack anonitimization.

There are far easier and faster ways to retrieve (more) reliable information about chars and their relations than to use these data. I have no doubt that some players with programming skills already use(d) them. I'll never help anyone on that point.

[o11c]

Make no mistake - I think there would be great benefit in releasing the full logs. After all, it *is* just a game, and people should not expect us to not try to improve the game. If anyone can actually be harmed in real life by this, they need to seriously rethink their internet identity practices.

And we certainly have the *right* to do anything we want, including public release, with all the data people give to us (except the email addresses and IP addresses). Of course, just because we can do something doesn't necessarily.

Despite the obvious benefits, remember that I (and, I think, others in the TMWC) tend to try to make the decisions that will make the least number of people angry, especially since the memory of our nasty breakup with Platyna is still so fresh.
We're at a point that we have more potential than ever before, but we aren't fully aware of the consequences of new things, and none of us developers have nearly as much time as we'd like to investigate fully ... hm, I didn't intend for this paragraph to become an argument for releasing the data so that other people can do that part ...

If I had my way, I would spend all my time deep in the code where none but the most daring can survive. But I keep getting dragged into issues like this, and I do what I can because somebody has to. I'm definitely happy with the way the TMWC ended up, spreading the power enough that I don't have to be a core part of the more "global" decisions.

So anyway - I hereby abstain from the TMWC's decision making process on this particular topic. I've added a few things to the discussion, take what you will.

Edit: oh, and Nard - could you stop double-posting?

[bell chick]

i dont see why armor needs to be reduced. all warriors really have is defense. and even at that my character cant handkle more than 2 skeletens without multi tapping chicken hoping that healing 2500 hp will be enough for survival. why does it need to be worse? but i digress. not the proper discussion

[Nard]

What makes me rather upset is that contributors continue to confuse RL privacy and these data. I trust administrators enough to have no doubt that personal data will not be shared with anyone.
Also it seems that most people (including developers) do not have any idea of the kind of information you can retrieve from statistics even if we all have a daily experience of the partial or wrong interpretations (not to say mistakes or lies) that our economists, politicians draw of the data we have. I am not surprised by that after having taught that topic for about 30 years to 60-120 students per year, and I would be happy help to better understand it. I am more than surprised that people (including persons who are supposed to have ascientific culture) can set strong affirmations on a topic where they are obviously rather ignorant. Now I cannot do anything for anyone who cannot imagine to be wrong. I don't need any stats to draw this conclusion.

My vote: Make opt out possible even for online list . Keep these data secret or better do not log them (just in case...)

[blackrazor]

I chose the first option, "full and ongoing", because any other option that limits the data in a non-random fashion will just skew the results. The data-points in a statistically solid data-set don't get to vote on their inclusion or exclusion. Anything that allows people to consciously opt-in or opt-out, by virtue of their actions or choices, will have this effect, in my opinion.

As others have said, I also draw a distinction between the real-life privacy of the human behind the pixels (which includes email address, IP address, chats) and the privacy of their pixel toon in an online video game (which includes combats, trades, interactions with NPCs, etc.), but I also recognize that not everyone will see this distinction in the same way.

And lastly, as others have also said, even with a valid data-set, it is quite possible to arrive at different conclusions based on pre-conceived bias, so I question the usefulness of this entire exercise. Regarding statistics, some of my favourite sayings are:

Numbers don't lie; humans lie using numbers.

Statistics is the method whereby outrageous textual falsehoods are converted into believable numeric ones.

This whole topic seems better decided by academic analysis, and not a vote. It's like deciding to vote on whether to use for-loops or while-loops in your code, in my opinion. Naturally, that's just my opinion.

[Nard]

Numbers don't lie; humans lie using numbers.

Statistics is the method whereby outrageous textual falsehoods are converted into believable numeric ones.

Sadly, that' often true, but the most frequent, as in many other domains (including human and "exact" sciences) is that stats users forget the underlying hypothesis of the model they use, and extend the answers beyond the limits that the tool allows.

The only kind of answer that stats can give us is: The 2 axis where the data spread the most are ..., the best linear model to fit these data is...., the 1% richest among the accounts own the same as the 40% poorest (probably far more here as a lot of accounts have no active characters or no character at all); I have no reason, with a risk of 5% to reject the hypothesis blablabla if I can suppose that the parameter Is approximatively gaussian. 5% or even 1% is very high, it happened often here that players were in this case, for Forest bow, fur boots or book pages: nearly 225,000 accounts have been created on this server since 2005, so it is certain (I should say almost sure in probability, or better very likely) that the improbable happens ...

btw: during Easter Event period, over than 1000 accounts have been created! I saw only about 20 chars among them. Even if I was not present a lot, this is weird. :/

[Freeyorp101]

(01:42:45) < Freeyorp> Jaxad0127: Most did, it's quite popular given that it's a reliable spawn location for clover
(01:42:51) * Jaxad0127 slaps meway for breaking github
(01:43:06) < Jaxad0127> Freeyorp: we should increase its timer.
(01:43:15) * meway probably did bork github.
(01:43:15) < Freeyorp> Jaxad0127: Did you see what I was working on for combat logs?
(01:43:23) < Jaxad0127> No.
(01:43:39) < Freeyorp> http://forums.themanaworld.org/viewtopi ... 12&t=17295
(01:44:03) < Jaxad0127> How about an invisible, unnamed, stationery monster?
(01:44:05) < meway> github.com works
(01:44:37) < Freeyorp> A more up to date screenshot: http://imagebin.org/256137
(01:45:23) < Freeyorp> It's really fast too, it takes a mere fraction of a second to apply new filters even on my little laptop
(01:45:55) < Jaxad0127> Red slime has a supprisingly large pie slice.
(01:47:07) < Jaxad0127> What are the axis' on the EXP per map graph?
(01:47:30) < Freeyorp> job exp versus base exp
(01:47:43) < Jaxad0127> We still award job exp?
(01:47:48) < Freeyorp> I wanted to see how closely they related
(01:48:02) < Freeyorp> Yes
(01:48:09) < Freeyorp> (Were they used for skill-pools?)
(01:48:23) < Freeyorp> I thought they were, but now I'm not so sure, hm.
(01:48:42) < Jaxad0127> How recent is that data view?
(01:48:44) < Freeyorp> But it's all shiny and interactive
(01:48:46) < Jaxad0127> What is the tool written in?
(01:49:03) < Freeyorp> See the graph down the bottom, instance breakdown by date
(01:49:09) < Freeyorp> It's all facets of the same dataset
(01:49:14) < Freeyorp> And javascript
(01:49:15) < Jaxad0127> That doesn't include month or year.
(01:49:29) < Freeyorp> Weird, it used to
(01:49:30) < Freeyorp> Bug!
(01:49:34) < Freeyorp> March 2013
(01:50:04) < Jaxad0127> Is the data restricted by the level thing in the upper right?
(01:50:11) < Freeyorp> Yes
(01:50:22) < Jaxad0127> THat's still a ton of red slimes.
(01:50:22) < Freeyorp> You can interactively restrict any of these facets at once
(01:50:39) < Freeyorp> It is fantastic
(01:50:50) < Jaxad0127> All processing is on the client side?
(01:51:07) < Freeyorp> I'm mesmerised by watching stat allocations change as you slide the brush on the level graph
(01:51:18) < Freeyorp> The stat correlation matrix is awesome
(01:51:19) < Freeyorp> Yes
(01:51:20) * Jaxad0127 wants
(01:51:28) < Freeyorp> It's all done on the client side in real time
(01:51:30) < Jaxad0127> Can we host it on the site?
(01:51:43) < Freeyorp> http://freeyorp.github.com/manavis/
(01:51:51) < Jaxad0127> How are you supposed to read the stat matrix?
(01:52:04) < Freeyorp> I may need to be push the gh-pages branch for that page to update, hang on
(01:52:16) < Freeyorp> Origin is 0,0
(01:52:25) < Freeyorp> Higher and further to the right is more
(01:52:33) < Jaxad0127> Now, where are those combat logs?
(01:52:53) < Freeyorp> So you get to see relations between each stat pairings
(01:53:06) < Freeyorp> Mainly because trying to graph 6D data directly is brain bending
(01:53:14) < Jaxad0127>
(01:53:30) < Freeyorp> This is why it's all symmetric, as agi-str and str-agi are quite related
(01:53:42) < Jaxad0127> So, origin is (0,0), but where is it?
(01:53:50) < Freeyorp> And things like str-str will only show relations within str
(01:54:02) < Freeyorp> Bottom left, same as with all other charts
(01:54:12) < Freeyorp> There are 36 subcharts
(01:54:19) < Jaxad0127> So, origin of the pi charts are in the bottom left? O_o
(01:54:20) < Freeyorp> Each with their own origin
(01:54:22) < Jaxad0127> Those are weird pies.
(01:54:25) < Freeyorp>
(01:54:30) < Jaxad0127> ;-P
(01:54:55) < Jaxad0127> So, darker is more pairings?
(01:55:10) < Freeyorp> But yeah, that's sort of the problem. Since it uses the map.logs, while the tool is public, the dataset is not.
(01:55:33) < Freeyorp> The thread I linked was about making a processed version of the map.logs public.
(01:55:33) < Jaxad0127> I thought the logs were scrubbed.
(01:55:52) < Freeyorp> There should be >4 years of logs around?
(01:56:43) < Jaxad0127> Where? I can't log into the current server.
(01:57:13) < Freeyorp> (01:55:10) < Freeyorp> But yeah, that's sort of the problem. Since it uses the map.logs, while the tool is public, the dataset is not.
(01:57:34) < Freeyorp> And from that thread, "Historially, the logs have been restricted, leaving analysis to the tiny intersection of server administrators and active developers, with rare exceptions, such as Fate."
(01:58:05) * Jaxad0127 read
(01:58:13) < Freeyorp> The thing is, the dataset could be used to stalk people if it's public. :/
(01:58:21) < Jaxad0127> We'll want a bit more processing of logins/logoffs to remove duplicates.
(01:58:26) < Freeyorp> Or suchlike.
(01:58:27) < Freeyorp> Yeah.
(01:58:28) < Jaxad0127> Do the lines identify chars?
(01:58:45) < Freeyorp> The breakdown by character ID does.
(01:59:30) < Jaxad0127> Send me one of the dailies and I'll look at scrubbing.
(01:59:45) < Freeyorp> I don't think I have access anymore either.
(01:59:58) < Freeyorp> I asked o11c for a days worth of logs to play with.
(02:00:16) < Jaxad0127> o11c: may I get some of the map logs? I want to work on a scrubber.
(02:01:41) < Jaxad0127> Once these become public, we could host an instance on the server that give you a list of available public logs.
(02:02:36) < Freeyorp> I guess that is the outcome. From the poll, a supermajority wanted continuous logs.
(02:02:47) < Freeyorp> (I didn't vote)
(02:03:02) -!- enchilado|web [82669e0f@defocus/yummy/enchilado] has joined #themanaworld-dev
(02:03:29) < Jaxad0127> For the stats, the last logoff per char should be used for best accurracy.
(02:05:10) < Freeyorp> Except where the logs end and they haven't logged off yet, for which the login stat should be used.
(02:05:21) < Freeyorp> But in any case, STAT changes are logged too.
(02:05:48) < Jaxad0127> Then keep track of those between login and log off.
(02:05:58) < Jaxad0127> Overwritting on logoff.
(02:06:11) < Freeyorp> I'm already parsing this, aren't I?
(02:06:41) < Jaxad0127> And put stats last so the preprocessor can keep the running values and add them at the end without having to go back over everything.
(02:06:54) < Jaxad0127> Though we would loose *when* stat changes happen.
(02:06:56) < Freeyorp> All login/logout info should be moved to the very start of the first logfile for which they're needed, with things like map position and time removed.
(02:07:17) < Jaxad0127> I think end because it'll simplify the processor.
(02:07:42) < Jaxad0127> The processor would be able to ech o cleansed lines for other data, and append the final stat values afterwards.
(02:07:54) < Freeyorp> Everything happens in a single forward parse. Tracking pending stat combinations in memory is cheaper than tracking pending experience gain instances in memory.
(02:08:12) < Freeyorp> Goodness, I'm tired today.
(02:08:25) < Freeyorp> Pardon my typing.
(02:08:37) < Jaxad0127> Mine has been worse today.
(02:10:24) < Freeyorp> Anyway, that thread discusses the sort of scrubbing we can do. Though for a continuous dataset, we can't play musical chairs with IDs between datasets, because there is no "between datasets".
(02:11:13) < Jaxad0127> Why do we need IDs in the scrubbed data?
(02:11:46) < Freeyorp> You need some identifier for characters, aliased or not, in order to link combat action to experience gain instances.
(02:12:05) < Freeyorp> The logs will only show KILLXP instances, not strictly what caused it.
(02:12:06) < Jaxad0127> Also, we could randomly assign an ID on login/char change. So you'd only know what someone did per session.
(02:12:32) < Freeyorp> And you also need to link STAT allocations to combat, and suchlike.
(02:12:40) < Freeyorp> Ah, that's an excellent idea!
(02:13:04) < Freeyorp> I guess we can play musical chairs after all.
(02:13:41) < meway> Do I have commit access?
(02:13:45) < Jaxad0127> We should complete remove trades between players and completely annonymize merchant interractions.
(02:13:52) < Jaxad0127> meway: to your stuff, yes.
(02:14:13) < meway> Jenalya: https://github.com/Meway/tmwa-server-data
(02:14:16) < Jaxad0127> Someone bought 5 apples from the fruit shop west of hurnscald.
(02:14:17) < Freeyorp> Yes, I mentioned that in my first post. Everything not relating to combat should be purged.
(02:14:34) < Jaxad0127> Some one sold 0 maggot slimes to Agatha.
(02:14:37) < Jaxad0127> *10
(02:14:40) < meway> Jenalya: Id just go outside and drop pickup.
(02:14:54) * meway sold 0 maggot slimes to agatha
(02:15:02) < Jaxad0127> Info on what transactions occurred is still useful.
(02:15:12) < Jaxad0127> We just don't need to know what players were involved.
(02:15:13) < Freeyorp> Is that even logged?
(02:15:27) < Jaxad0127> IDK.
(02:15:46) < Freeyorp> There really isn't that much that's logged. Even what equipment (aside from weapon) used is not logged.
(02:16:18) -!- enchilado|web [82669e0f@defocus/yummy/enchilado] has quit [Ping timeout: 245 seconds]
(02:16:23) < Jaxad0127> Adding merchant transactions shouldn't be hard.
(02:17:14) < Freeyorp> The question is whether it's warranted.
(02:17:34) < Jaxad0127> There is that.
(02:18:20) -!- enchilado|web [82669e0f@defocus/yummy/enchilado] has joined #themanaworld-dev
(02:18:44) < Jaxad0127> MadCamel1: how does this map log annonymization strategy sound: every tome a character is chose, it's actions are assigned a random ID.
(02:19:12) < Jaxad0127> MadCamel1: if a player plays with char A, then switches to char B, then switches back to char A, their actions would be under three different, random IDs.
(02:19:15) < Freeyorp> I'll update the thread. I'm not sure I can put together something coherent, do you mind if I quote the discussion here?
(02:19:26) < Jaxad0127> Nope.
(02:20:10) < Jaxad0127> This still allows a level of tracking.
(02:20:20) < Jaxad0127> Especially with mobs like the Pumkin Ghost.
(02:21:06) < Jaxad0127> Can we still completely anonymize login/log off stats and only leave increases tied tot he random char ID?
(02:21:09) -!- Kage_ is now known as Kage
(02:22:40) < Freeyorp> Yes, it should be possible to move everything to a random temporary login session ID.
(02:23:01) < Jaxad0127> I'm still owrried about login/logoff stats.
(02:23:14) < Jaxad0127> Do we need to tie those to char IDs at all?
(02:23:14) < Freeyorp> Remove any reference to location or time for them?
(02:23:28) < Jaxad0127> And character.
(02:23:32) < Freeyorp> Not to char IDs, but they need at least session IDs to link them to experience gain instances
(02:23:53) < Jaxad0127> WHich mean all char stats are trackable....
(02:23:54) < meway> https://github.com/Meway/tmwa-server-data sombody should give me commit access to tmw stuffs
(02:24:05) < Jaxad0127> meway: make a pull request.
(02:24:33) < Freeyorp> Hmm.
(02:24:49) < Kage> Freeyorp is back too?
(02:24:55) < Jaxad0127> Kage: he has been for a while.
(02:24:55) < Freeyorp> Oh, hello Kage!
(02:25:22) < Jaxad0127> Kage: where have *you* been?
(02:25:38) < Freeyorp> I round stat allocations in my tool, perhaps doing the same in the dataset would make things less unique and less trackable?
(02:25:39) < Jaxad0127> Freeyorp has been back for a few weeks from the looks of it. I've been back nearly a week.
(02:25:44) < Kage> Jaxad0127: I check the chat room every week or so
(02:25:53) < Jaxad0127> ....
(02:26:01) < Jaxad0127> Why so sparsely?
(02:26:13) < BronzeEagle> Because you weren't here, Jaxad0127
(02:26:19) < Jaxad0127>
(02:26:20) < Kage> partly...
(02:26:40) < Freeyorp> Frost offered me a replacement shell I think a month or so ago, so I've been around a little more lately.
(02:26:50) * meway is back too ! \\o/ ;D
(02:27:02) < Freeyorp> Anyway, what do you think about rounding stat allocations?
(02:27:24) < meway> -.- you three monooose <.<
(02:27:29) < meway> mongooos
(02:27:55) < Jaxad0127> How would that help?
(02:28:03) < meway> peace
(02:28:04) < Jaxad0127> meway: do you mean mongooses?
(02:28:17) < Freeyorp> With only coarse stat allocation, no location information, and no timestamps, linking characters between session IDs would be much harder.
(02:28:46) < Jaxad0127> I didn't think of cross login correllation.
(02:28:51) < Jaxad0127> That should help tremendously.
(02:29:39) < Kage> meway: you needed some packages installed on Kagenet?
(02:30:00) < Jaxad0127> Kage: can you install a sanity package for him?
(02:30:11) < meway> Kage: yes
(02:30:18) < Kage> meway: what?
(02:30:21) < Freeyorp> It should provide the same level of protection as the original "musical chairs" ID system.
(02:30:36) < Jaxad0127> Freeyorp: true.
(02:30:37) < meway> Kage: I forgot
(02:30:42) < Jaxad0127> Is that still in place?
(02:30:55) < Jaxad0127> Can we piggyback on it
(02:30:56) < Jaxad0127> ?
(02:31:17) < Freeyorp> Erm, that was a method, not an implementation.
(02:31:46) < Kage> That "Musical chairs ID system" was a nightmare...
(02:31:58) < Freeyorp> What?
(02:32:07) < Kage> On ManaServ
(02:32:13) < Jaxad0127> The presence of the inline list does complicate making those logs public.
(02:32:22) < Jaxad0127> *online
(02:32:25) < Freeyorp> I think we're talking about different things here, Kage
(02:32:42) < Kage> Remember the being ID allocator on ManaServ
(02:32:56) < Freeyorp> Well, if you move all login/logout to the start of the first applicable logfile, it's hard to link them.
(02:33:04) < Kage> That was the source of memory corruption for about 3 years until I rewrote it
(02:33:04) < Freeyorp> Bleh, yeah. :/
(02:33:21) < Jaxad0127> Freeyorp: you can still link activity to changes in the online list.
(02:33:48) < Jaxad0127> Do we need to track chars between maps?
(02:34:13) < Freeyorp> Considering the number of idle characters, and the fact that session IDs would be transient, that should be hard to do.
(02:34:39) < Freeyorp> The problem is the STAT assignment. It's logged when characters log in, log out, and when it changes.
(02:34:59) < Jaxad0127> The third one only needs to log deltas.
(02:35:14) < Jaxad0127> OR do we need everything?
(02:35:19) < Freeyorp> You could create a new ID on map change, but since one STAT set expires and another one comes into play at the same time, it doesn't need much.
(02:35:20) < meway> github is bokred again
(02:35:29) < Freeyorp> Deltas would just require even more information.
(02:35:33) < Jaxad0127> meway: stop breaking it.
(02:35:37) < Jaxad0127> !fault
(02:35:44) < Jaxad0127> Hm.. what was that command?
(02:35:50) < Freeyorp> What command?
(02:35:52) < Jaxad0127> !blame
(02:35:52) < elanore> It's meway's fault.
(02:35:54) < Kage> meway: I think you needed lib32stdc++6 package, its installed now
(02:35:55) < Jaxad0127> That one.
(02:36:01) < Freeyorp> Oh my.
(02:36:13) < meway> lol
(02:36:18) * meway is amused
(02:36:28) < Jaxad0127> !rq meway
(02:36:28) < elanore> * Meway wonders what this joke he didn't understand
(02:36:31) < Jaxad0127> lol
(02:36:37) < Jaxad0127> Appropriate again.
(02:36:41) < meway> lol
(02:37:05) * Kage wonders if Jaxad0127 is still a Java fanboy
(02:37:11) < meway> Jaxad0127: I can't request pull with no access.
(02:37:19) < meway> to github
(02:37:33) < Kage> meway: use the http source, not the git source
(02:37:42) < Freeyorp> Anyway, updating that thread now. It seems that's the majority decision, and I must say I'm a lot happier about being able to move things back to transient IDs.
(02:37:45) < Jaxad0127> meway: you have your clone in github. Push to that, then request a pull from there to the main repo.
(02:38:14) < Freeyorp> Got to keep everyone up to date.
(02:38:15) < Jaxad0127> Freeyorp: so, we do need to track map changes?
(02:38:24) < Jaxad0127> Kage: I work with it professionally.
(02:38:37) < Kage> Jaxad0127: let me guess, anadroid dev now?
(02:38:49) < Freeyorp> Jaxad0127: Location is logged with experience gain instances, map change itself is not inherently logged.
(02:38:59) < Jaxad0127> Kage: no.
(02:39:00) < meway> to github?
(02:39:26) < Jaxad0127> Freeyorp: I was wondering if the server could randomly assign an ID on map change. Would that break your tool?
(02:39:29) < meway> kage how am I suppose to request pull?
(02:39:43) < BronzeEagle> meway: say please, obviously
(02:39:57) < meway> !rq
(02:39:57) < elanore> <melkior> In fact, I'm always surprised when version control, well, just works.
(02:39:59) < Jaxad0127> meway: after pushing your chagnes to *your* github, request a pull from *your* github to the main github.
(02:40:02) < Jaxad0127> It's in the interface.
(02:40:21) * meway forgot how to do stuff.
(02:40:27) < Jaxad0127> git push
(02:40:31) < Freeyorp> Jaxad0127: You could, and it wouldn't break things. But you'd still need to log IDs. And since you don't know when that's needed until after the fact, you'd need to start inherently logging map change, and scrub out unneeded ones before public release.
(02:40:34) < Jaxad0127> Everything else is in the web interface.
(02:40:40) < meway> git push git://github.com/themanaworld/tmwa.git
(02:40:40) < Freeyorp> That might make things even more transient.
(02:40:41) < meway> ?
(02:41:00) < Jaxad0127> meway: did you clone from your github or the main one?
(02:41:00) < meway> remote host is down <.<
(02:41:11) < Jaxad0127> meway: please stop breaking github.
(02:41:27) < Freeyorp> Though I guess you could already replace IDs with the scrubber with the information logged already, without needing to log more?
(02:41:31) * meway didn't do it <.<
(02:41:42) < Jaxad0127> 20:41 < Jaxad0127> meway: did you clone from your github or the main one?
(02:42:16) < meway> I had to copy the zip from websource and put it in mine.
(02:42:33) < Freeyorp> I'm liking this. We can replace things with random IDs even more transient than I had originally planned.
[...]
(02:47:53) < Freeyorp> Posting now for real. You all keep coming up with awesome ideas that delay my post.

tl;dr:

The supermajority of people seem to want ongoing public datasets, which it looks like we're probably going with (feel free to discuss things further!); Jaxad thought of a way to scrub the datasets to make IDs even more transient; meway still cannot into github

The updated screenshot mentioned above is also attached here in case people are curious:

manavis.png (176.8 KiB) Viewed 2283 times

---Freeyorp

[Jaxad0127]

After more discussion, we think we can still make the tool work if we change the identifier used for each character every time they log in, change characters, move between maps, and after X minutes of inactivity.

Note that these logs would just include combat, experience gain, and stat changes. Not too much else even gets logged, and it would all be scrubbed before the logs are made public.

[radiant]

Not that I mind, but outliers really do stand out on the chart; from the visible data it appears that I'm single-handedly responsible for every "instance" of an event in those logs where luck is greater than 90. Luck in the 80s also appears to have a unique contributor, though one who's somewhat more active than I am.

[Crush]

Can someone explain how to read the "instance breakdown by Stat allocation" diagram?

[Freeyorp101]

It's a trellis chart, a correlation matrix; it consists of 36 subcharts, each describing the relationship between its associated two attributes.

It's the most incomplete chart at the moment; I still haven't finished drawing in all the chart separation lines or any of the scales - 0 to the left and bottom and 99 to the top and right for each subchart.

Because each attribute is listed in order, the chart has a line of symmetry - str-agi and agi-str will reflect each other. Right on the diagonal, relating each attribute to itself, we can use the fact that each column only will have one entry to see the relative allocation intensities within the attribute.

For example, in the screenshot above, we can see that there is absolutely no one (at least within the active filter of levels 83-99) that has not put a substantial number of points into either agility or vitality.
As you slide the range of levels filtered up, you can see a sort of an arc moving diagonally up and right.

It's not limited to agi-vit either; you can make more interesting inferences about everything when sliding the base level filter up and down, and seeing where the hotspots shift as characters get more and more stat points to allocate.

The same goes for any set of filters; see what's popular over in the graveyard, 027-1 (which is more popular than any other map for experience gain by an order of magnitude for both level and job experience), see what people like to attack with bows, or swords, or spells.

I'm sure I haven't come even close to finding out everything about this dataset, and I doubt that I'll be able to do so alone. Many eyes should make light work; and hopefully a bit of fun while we're at it.

---Freeyorp