Page 1 of 1

change the purge account protocols

Posted: 30 Sep 2012, 22:32
by blackrazor
Currently, inactive accounts are routinely purged from the system every year or so, all accounts inactive for over six months are deleted at that time. The reason is to maintain the efficiency of the login server, I am told.

This can be problematic, as players that have put a lot of time and effort into their accounts might have taken a break from the game. Maybe they went off to try other projects, or maybe they got sick, or lost computer access, or got military deployment overseas. There can be any number of reasons why they stop playing, and when they return, they are saddened to find that their Mana World player account is purged, forever gone.

I recommend the following changes to the purge protocol:

When the purge is run, it makes a list of inactive accounts, as usual. Any inactive account with all characters under level 60 can be (hopefully) safely deleted. Level 60 is an arbitrary number, and can be adjusted as needed. If an account has a character at or above that level, then that account is moved into a secondary database that is not actively referenced by the login server (and thus does not impair its efficiency), but it is not deleted either. If that player some day returns (as many high levels probably eventually do), they can request an automated transfer from the secondary database to the main playing database. Depending on the logistics involved, it can either happen within minutes, or within 24 hours as part of an automated batch process.

There was another game I once enjoyed that used this sort of 2-tier account database system, that allowed the actively played database to be kept small, but didn't permanently delete older accounts either. I had hoped that this system might work well here also, and so I make this suggestion.

Re: change the purge account protocols

Posted: 01 Oct 2012, 00:32
by Chicka-Maria
The last purge was from whoever didnt log on for a year, which i think is pretty fair. Its not hard to log on once every 6 months even if they did take a break.

Regards,

Re: change the purge account protocols

Posted: 01 Oct 2012, 01:19
by Frost
blackrazor wrote:Currently, inactive accounts are routinely purged from the system every year or so, all accounts inactive for over six months are deleted at that time. The reason is to maintain the efficiency of the login server, I am told.
Thus far, they've been purged after the server started having serious problems. Accounts that were completely unused for at least a year were removed.
To be specific, the char-server process saves the data file every few minutes. It parses the data, sorts it, and does various other CPU-intensive work. If the time required to save the data file becomes longer than the time allowed by the OS for the process to terminate gracefully (SIGTERM), then we get incomplete save files, which leads to loss of accounts and items whenever the server crashes or is stopped. Note that this doesn't include performance concerns, merely basic functionality.
This can be problematic, as players that have put a lot of time and effort into their accounts might have taken a break from the game. Maybe they went off to try other projects, or maybe they got sick, or lost computer access, or got military deployment overseas. There can be any number of reasons why they stop playing, and when they return, they are saddened to find that their Mana World player account is purged, forever gone.
I sympathize, but we have to draw the line somewhere in order to keep the server functioning. A year is a long time away from a social MMORPG.
I recommend the following changes to the purge protocol:

When the purge is run, it makes a list of inactive accounts, as usual. Any inactive account with all characters under level 60 can be (hopefully) safely deleted.
This could be done.
If an account has a character at or above that level, then that account is moved into a secondary database that is not actively referenced by the login server (and thus does not impair its efficiency), but it is not deleted either.
All character data is stored in one flat (text) file, not a database. This file is read when the server loads, and is merely overwitten whenever the data is auto-saved. Data is either in that file at load time, or it is not. Of course, the purge script could generate a separate file of "purged accounts and characters."
If that player some day returns (as many high levels probably eventually do), they can request an automated transfer from the secondary database to the main playing database. Depending on the logistics involved, it can either happen within minutes, or within 24 hours as part of an automated batch process.
Note that there is no tool to separate or merge data into the running server; any changes would be done with a text editor after shutdown. What you suggest would require shutting down the game server for perhaps 15 minutes.

Although I agree with the purpose of this proposal, it would be a serious pain in the ass to perform with the current software. It's not clear to me, at least, that it's worth the cost to regain a few players who left the game years ago.

[edited for brevity]

Re: change the purge account protocols

Posted: 01 Oct 2012, 18:12
by o11c
One of the long-term plans I have is to fix the storage backend to avoid the technical problems here. But that will be a long time coming.

Once that happens, there will be one final period of one year (or so), and then old accounts can be deleted for the last time. (the reason for this is that I expect that part of the conversion process will require the player to log in)

Again: this will be a long time coming. There are no plans for changing the purge protocol in the countable future.

Re: change the purge account protocols

Posted: 17 Oct 2012, 15:12
by Nard
From an absolute point and technical point of view all the arguments you develop are understandable. But we miss the data size and rates information to build a realistic image of the problem.

About the size problem some questions I asked some time ago are still left unanswered : how is it possible that the rate of account creation stays constant while every one who plays can see the number of of players decrease?
I also pointed out in those posts (http://forums.themanaworld.org/viewtopi ... 12&t=15846) that the character per account ratio is under 1 and had decreased with last purge, thus it had deleted more players than empty accounts. If you add the fact that many players have more than a char per account (I have a mean of 4) we can easily imagine the weight these empty accounts represent in the database. They are likely to be an important cause of the problems you have to deal with. Why not suppress these emtpy accounts first and see the result? In my opinion they almost are a kind of DOS attack on the server thus empty accounts could be watched and wiped after a week or 15 days unused.
a second "line" that you could draw is to wipe accounts that have been used less than 1,2,3...weeks and unused after, these are obviously player who didn't like the game.
Frost wrote:All character data is stored in one flat (text) file, not a database. This file is read when the server loads, and is merely overwitten whenever the data is auto-saved. Data is either in that file at load time, or it is not. Of course, the purge script could generate a separate file of "purged accounts and characters."
This could be made easier if the database was sorted by last date of use than by ID (or creation date which is almost the same)
The Database engine seems very old and outdated. "Modern" ones , even those which have to deal with huge single sequential files (sound, video, CAD,...) operate virtual and buffered editing and deal only with database changes which they store incrementally. When needed, backup and compaction occurs in background or in service time, while off line. If it causes so many problems, it's replacement is critical in the project, I wished I could help for that :(. It could be a very nice project for computer science students in university.

Maybe a year (some purges occured with 6 months of inactivity) for a mmorpg, but having old friends back is nice to active ones. Life is not always "fair" and can oblige you to quit without warning and a year is not that long in real life.

Re: change the purge account protocols

Posted: 17 Oct 2012, 15:57
by Frost
Nard wrote:thus empty accounts could be watched and wiped after a week or 15 days unused.
a second "line" that you could draw is to wipe accounts that have been used less than 1,2,3...weeks and unused after, these are obviously player who didn't like the game.
...
This could be made easier if the database was sorted by last date of use than by ID (or creation date which is almost the same)
The Database engine seems very old and outdated. "Modern" ones , even those which have to deal with huge single sequential files (sound, video, CAD,...) operate virtual and buffered editing and deal only with database changes which they store incrementally.
Those are great ideas. I agree with your suggestion that we focus more narrowly on obviously un-used accounts, as well as your criteria to identify them.

Unfortunately, server data is not stored in a database.
  • Account data including last login are stored in the flat file account.txt. We can reference tmwAthena account ID number.
  • Character data are stored in the flat file athena.txt. We can reference tmwAthena account ID number.
  • Account creation date is stored in the MySQL database back-end to the web registration. The login name can be cross-referenced in account.txt.
A script could parse the flat files and even query the database and create a file with any structure desired. It could generate a script to purge the identified accounts. I'm not skilled enough to write that tool.

Re: change the purge account protocols

Posted: 17 Oct 2012, 19:31
by Crush
Aggressively purging empty accounts would make sense, because when they get deleted, nothing of value is lost. When someone wants the account back, they can just re-register it.

Re: change the purge account protocols

Posted: 18 Oct 2012, 05:08
by Nard
Frost wrote:.
Unfortunately, server data is not stored in a database.
  • Account data including last login are stored in the flat file account.txt. We can reference tmwAthena account ID number.
  • Character data are stored in the flat file athena.txt. We can reference tmwAthena account ID number.
  • Account creation date is stored in the MySQL database back-end to the web registration. The login name can be cross-referenced in account.txt.
The account server must know wich chars are linked to an account, I don't understand where this information is :?
A script could parse the flat files and even query the database and create a file with any structure desired. It could generate a script to purge the identified accounts. I'm not skilled enough to write that tool.
A script would be a good first approach, but account management tools shoud be intergrated into the account server user (administrator) interface in a longer term. A game admin should be able to get quickly an overview of statistical player information. This important not only to be able to take administrative decisions but also for outside communication: advertizing....
When I am better skilled in python I will try to help; this will not be tomorrow unfortunately.

Re: change the purge account protocols

Posted: 18 Oct 2012, 10:28
by Freeyorp101
Nard wrote:[...]account management tools shoud be intergrated into the account server user (administrator) interface in a longer term.
We have a non-bash-fu interface now?

---Freeyorp

Re: change the purge account protocols

Posted: 19 Oct 2012, 11:43
by Nard
:shock: I noticed that in purger.java: :shock:

Code: Select all

catch (ParseException e) {
                            // Ignore accounts that haven't been used yet
                            if (fields[3].equals("-")) {
                                activeAccounts.add(fields[0]);
                                copy = true;
So if I understand well unused accounts are removed except if they have never been used ?!

Re: change the purge account protocols

Posted: 19 Oct 2012, 17:46
by o11c
confirmed that field[3] == "-" happens for newly created accounts.