change the purge account protocols

Got something on your mind about the project? This is the correct place for that.


Forum rules

This forum is for feature requests, content changes additions, anything not a Bug in the software.
Please report all bugs on the Support Forums

Post Reply
blackrazor
Warrior
Warrior
Posts: 332
Joined: 18 Oct 2007, 13:38

change the purge account protocols

Post by blackrazor »

Currently, inactive accounts are routinely purged from the system every year or so, all accounts inactive for over six months are deleted at that time. The reason is to maintain the efficiency of the login server, I am told.

This can be problematic, as players that have put a lot of time and effort into their accounts might have taken a break from the game. Maybe they went off to try other projects, or maybe they got sick, or lost computer access, or got military deployment overseas. There can be any number of reasons why they stop playing, and when they return, they are saddened to find that their Mana World player account is purged, forever gone.

I recommend the following changes to the purge protocol:

When the purge is run, it makes a list of inactive accounts, as usual. Any inactive account with all characters under level 60 can be (hopefully) safely deleted. Level 60 is an arbitrary number, and can be adjusted as needed. If an account has a character at or above that level, then that account is moved into a secondary database that is not actively referenced by the login server (and thus does not impair its efficiency), but it is not deleted either. If that player some day returns (as many high levels probably eventually do), they can request an automated transfer from the secondary database to the main playing database. Depending on the logistics involved, it can either happen within minutes, or within 24 hours as part of an automated batch process.

There was another game I once enjoyed that used this sort of 2-tier account database system, that allowed the actively played database to be kept small, but didn't permanently delete older accounts either. I had hoped that this system might work well here also, and so I make this suggestion.
User avatar
Chicka-Maria
TMW Adviser
TMW Adviser
Posts: 1562
Joined: 19 Feb 2010, 02:10
Location: Internet

Re: change the purge account protocols

Post by Chicka-Maria »

The last purge was from whoever didnt log on for a year, which i think is pretty fair. Its not hard to log on once every 6 months even if they did take a break.

Regards,
Yubaba
TMWC Member of The Mana World
Leader of The Mana Empire (TME)
[19:41] Ladysugar: he told me to push a setzer up his rear
www.deviantart.com/comfycheeks - Old Deviant Art
William James wrote:Act as If what you do make's a difference, because It does.
Frost
TMW Adviser
TMW Adviser
Posts: 851
Joined: 09 Sep 2010, 06:20
Location: California, USA

Re: change the purge account protocols

Post by Frost »

blackrazor wrote:Currently, inactive accounts are routinely purged from the system every year or so, all accounts inactive for over six months are deleted at that time. The reason is to maintain the efficiency of the login server, I am told.
Thus far, they've been purged after the server started having serious problems. Accounts that were completely unused for at least a year were removed.
To be specific, the char-server process saves the data file every few minutes. It parses the data, sorts it, and does various other CPU-intensive work. If the time required to save the data file becomes longer than the time allowed by the OS for the process to terminate gracefully (SIGTERM), then we get incomplete save files, which leads to loss of accounts and items whenever the server crashes or is stopped. Note that this doesn't include performance concerns, merely basic functionality.
This can be problematic, as players that have put a lot of time and effort into their accounts might have taken a break from the game. Maybe they went off to try other projects, or maybe they got sick, or lost computer access, or got military deployment overseas. There can be any number of reasons why they stop playing, and when they return, they are saddened to find that their Mana World player account is purged, forever gone.
I sympathize, but we have to draw the line somewhere in order to keep the server functioning. A year is a long time away from a social MMORPG.
I recommend the following changes to the purge protocol:

When the purge is run, it makes a list of inactive accounts, as usual. Any inactive account with all characters under level 60 can be (hopefully) safely deleted.
This could be done.
If an account has a character at or above that level, then that account is moved into a secondary database that is not actively referenced by the login server (and thus does not impair its efficiency), but it is not deleted either.
All character data is stored in one flat (text) file, not a database. This file is read when the server loads, and is merely overwitten whenever the data is auto-saved. Data is either in that file at load time, or it is not. Of course, the purge script could generate a separate file of "purged accounts and characters."
If that player some day returns (as many high levels probably eventually do), they can request an automated transfer from the secondary database to the main playing database. Depending on the logistics involved, it can either happen within minutes, or within 24 hours as part of an automated batch process.
Note that there is no tool to separate or merge data into the running server; any changes would be done with a text editor after shutdown. What you suggest would require shutting down the game server for perhaps 15 minutes.

Although I agree with the purpose of this proposal, it would be a serious pain in the ass to perform with the current software. It's not clear to me, at least, that it's worth the cost to regain a few players who left the game years ago.

[edited for brevity]
You earn respect by how you live, not by what you demand.
-unknown
User avatar
o11c
Grand Knight
Grand Knight
Posts: 2262
Joined: 20 Feb 2011, 21:09
Location: ^ ^

Re: change the purge account protocols

Post by o11c »

One of the long-term plans I have is to fix the storage backend to avoid the technical problems here. But that will be a long time coming.

Once that happens, there will be one final period of one year (or so), and then old accounts can be deleted for the last time. (the reason for this is that I expect that part of the conversion process will require the player to log in)

Again: this will be a long time coming. There are no plans for changing the purge protocol in the countable future.
Former programmer for the TMWA server.
User avatar
Nard
Knight
Knight
Posts: 1113
Joined: 27 Jun 2010, 12:45
Location: France, near Paris

Re: change the purge account protocols

Post by Nard »

From an absolute point and technical point of view all the arguments you develop are understandable. But we miss the data size and rates information to build a realistic image of the problem.

About the size problem some questions I asked some time ago are still left unanswered : how is it possible that the rate of account creation stays constant while every one who plays can see the number of of players decrease?
I also pointed out in those posts (http://forums.themanaworld.org/viewtopi ... 12&t=15846) that the character per account ratio is under 1 and had decreased with last purge, thus it had deleted more players than empty accounts. If you add the fact that many players have more than a char per account (I have a mean of 4) we can easily imagine the weight these empty accounts represent in the database. They are likely to be an important cause of the problems you have to deal with. Why not suppress these emtpy accounts first and see the result? In my opinion they almost are a kind of DOS attack on the server thus empty accounts could be watched and wiped after a week or 15 days unused.
a second "line" that you could draw is to wipe accounts that have been used less than 1,2,3...weeks and unused after, these are obviously player who didn't like the game.
Frost wrote:All character data is stored in one flat (text) file, not a database. This file is read when the server loads, and is merely overwitten whenever the data is auto-saved. Data is either in that file at load time, or it is not. Of course, the purge script could generate a separate file of "purged accounts and characters."
This could be made easier if the database was sorted by last date of use than by ID (or creation date which is almost the same)
The Database engine seems very old and outdated. "Modern" ones , even those which have to deal with huge single sequential files (sound, video, CAD,...) operate virtual and buffered editing and deal only with database changes which they store incrementally. When needed, backup and compaction occurs in background or in service time, while off line. If it causes so many problems, it's replacement is critical in the project, I wished I could help for that :(. It could be a very nice project for computer science students in university.

Maybe a year (some purges occured with 6 months of inactivity) for a mmorpg, but having old friends back is nice to active ones. Life is not always "fair" and can oblige you to quit without warning and a year is not that long in real life.
"The language of everyday life is clogged with sentiment, and the science of human nature has not advanced so far that we can describe individual sentiment in a clear way." Lancelot Hogben, Mathematics for the Million.
“There are two motives for reading a book; one, that you enjoy it; the other, that you can boast about it.” Bertrand Russell, Conquest of Happiness.
"If you optimize everything, you will always be unhappy." Donald Knuth.
Frost
TMW Adviser
TMW Adviser
Posts: 851
Joined: 09 Sep 2010, 06:20
Location: California, USA

Re: change the purge account protocols

Post by Frost »

Nard wrote:thus empty accounts could be watched and wiped after a week or 15 days unused.
a second "line" that you could draw is to wipe accounts that have been used less than 1,2,3...weeks and unused after, these are obviously player who didn't like the game.
...
This could be made easier if the database was sorted by last date of use than by ID (or creation date which is almost the same)
The Database engine seems very old and outdated. "Modern" ones , even those which have to deal with huge single sequential files (sound, video, CAD,...) operate virtual and buffered editing and deal only with database changes which they store incrementally.
Those are great ideas. I agree with your suggestion that we focus more narrowly on obviously un-used accounts, as well as your criteria to identify them.

Unfortunately, server data is not stored in a database.
  • Account data including last login are stored in the flat file account.txt. We can reference tmwAthena account ID number.
  • Character data are stored in the flat file athena.txt. We can reference tmwAthena account ID number.
  • Account creation date is stored in the MySQL database back-end to the web registration. The login name can be cross-referenced in account.txt.
A script could parse the flat files and even query the database and create a file with any structure desired. It could generate a script to purge the identified accounts. I'm not skilled enough to write that tool.
You earn respect by how you live, not by what you demand.
-unknown
User avatar
Crush
TMW Adviser
TMW Adviser
Posts: 8046
Joined: 25 Aug 2005, 16:08
Location: Germany

Re: change the purge account protocols

Post by Crush »

Aggressively purging empty accounts would make sense, because when they get deleted, nothing of value is lost. When someone wants the account back, they can just re-register it.
  • former Manasource Programmer
  • former TMW Pixel artist
  • NOT a game master

Please do not send me any inquiries regarding player accounts on TMW.


You might have heard a certain rumor about me. This rumor is completely false. You might also have heard the other rumor about me. This rumor is 100% accurate.
User avatar
Nard
Knight
Knight
Posts: 1113
Joined: 27 Jun 2010, 12:45
Location: France, near Paris

Re: change the purge account protocols

Post by Nard »

Frost wrote:.
Unfortunately, server data is not stored in a database.
  • Account data including last login are stored in the flat file account.txt. We can reference tmwAthena account ID number.
  • Character data are stored in the flat file athena.txt. We can reference tmwAthena account ID number.
  • Account creation date is stored in the MySQL database back-end to the web registration. The login name can be cross-referenced in account.txt.
The account server must know wich chars are linked to an account, I don't understand where this information is :?
A script could parse the flat files and even query the database and create a file with any structure desired. It could generate a script to purge the identified accounts. I'm not skilled enough to write that tool.
A script would be a good first approach, but account management tools shoud be intergrated into the account server user (administrator) interface in a longer term. A game admin should be able to get quickly an overview of statistical player information. This important not only to be able to take administrative decisions but also for outside communication: advertizing....
When I am better skilled in python I will try to help; this will not be tomorrow unfortunately.
User avatar
Freeyorp101
Archivist Prime
Archivist Prime
Posts: 765
Joined: 04 Nov 2008, 09:17
Location: New Zealand

Re: change the purge account protocols

Post by Freeyorp101 »

Nard wrote:[...]account management tools shoud be intergrated into the account server user (administrator) interface in a longer term.
We have a non-bash-fu interface now?

---Freeyorp
(09:58:17) < tux9th> Freeyorp: your sig on the forums is kind of outdated
User avatar
Nard
Knight
Knight
Posts: 1113
Joined: 27 Jun 2010, 12:45
Location: France, near Paris

Re: change the purge account protocols

Post by Nard »

:shock: I noticed that in purger.java: :shock:

Code: Select all

catch (ParseException e) {
                            // Ignore accounts that haven't been used yet
                            if (fields[3].equals("-")) {
                                activeAccounts.add(fields[0]);
                                copy = true;
So if I understand well unused accounts are removed except if they have never been used ?!
"The language of everyday life is clogged with sentiment, and the science of human nature has not advanced so far that we can describe individual sentiment in a clear way." Lancelot Hogben, Mathematics for the Million.
“There are two motives for reading a book; one, that you enjoy it; the other, that you can boast about it.” Bertrand Russell, Conquest of Happiness.
"If you optimize everything, you will always be unhappy." Donald Knuth.
User avatar
o11c
Grand Knight
Grand Knight
Posts: 2262
Joined: 20 Feb 2011, 21:09
Location: ^ ^

Re: change the purge account protocols

Post by o11c »

confirmed that field[3] == "-" happens for newly created accounts.
Former programmer for the TMWA server.
Post Reply