Forming a beta testing team

Content and general development discussion, including quest scripts and server code. TMW Classic is a project comprising the Legacy tmwAthena server & the designated improved engine server based on evolHercules.


Forum rules

This forum houses many years of development, tracing back to some of the earliest posts that exist on the board.

Its current use is for the continued development of the server and game it has always served: TMW Classic.

User avatar
Nard
Knight
Knight
Posts: 1113
Joined: 27 Jun 2010, 12:45
Location: France, near Paris

Re: Forming a beta testing team

Post by Nard »

Jenalya wrote:I'd suggest the feedback forum for bug reports rather than playertalk. Topics about testing something specific fit to content development.
This is a good idea, we can see later if there is a need for more specific one.

Note: I am already a Quassel user. I could not find a way to keep it on line while my son uses this desktop. I still have to buy 2 computers to be able to do it.:)
"The language of everyday life is clogged with sentiment, and the science of human nature has not advanced so far that we can describe individual sentiment in a clear way." Lancelot Hogben, Mathematics for the Million.
“There are two motives for reading a book; one, that you enjoy it; the other, that you can boast about it.” Bertrand Russell, Conquest of Happiness.
"If you optimize everything, you will always be unhappy." Donald Knuth.
User avatar
Wombat
TMW Adviser
TMW Adviser
Posts: 1532
Joined: 08 Aug 2008, 16:31

Re: Forming a beta testing team

Post by Wombat »

Nard wrote:impossible for me, some players cannot even play while their browser is open.
How is it impossible?
Nard wrote:I don't agree with you, and go on thinking, that a specific forum would be more efficient, if not to you (you would have one more to read) at least to testers and bug reporters.
I recommend using freeforums if you can't work with what we have offered.
Current character is "Abolish".
User avatar
Crush
TMW Adviser
TMW Adviser
Posts: 8046
Joined: 25 Aug 2005, 16:08
Location: Germany

Re: Forming a beta testing team

Post by Crush »

I think this whole discussion leads nowhere.

All arguments presented so far (including mine) are only based on theory. How can we decide what's the best way to organize a QA process when nobody here worked as or with this new beta testing team yet?

I think we should try out how this whole beta testing team works out in practice, collect some experiences, and after you actually reviewed a few patches we try to figure out the best processes and what tools we really need for them. Currently we are just running into pointless arguing which is dangerously close to making this QA initiative fail before it even started, because we are all at each others throats before we even accomplished anything.
  • former Manasource Programmer
  • former TMW Pixel artist
  • NOT a game master

Please do not send me any inquiries regarding player accounts on TMW.


You might have heard a certain rumor about me. This rumor is completely false. You might also have heard the other rumor about me. This rumor is 100% accurate.
User avatar
o11c
Grand Knight
Grand Knight
Posts: 2262
Joined: 20 Feb 2011, 21:09
Location: ^ ^

Re: Forming a beta testing team

Post by o11c »

I do support the creation of a new forum for testing, under Community, not tmwAthena Development. I do think that we need a player review/testing system.

I wish to repeat that mantis is not effective in its current form. That said, it's not really player-oriented in any form.

Trello might work, but it's still less likely to draw in new testers than a forum would be.

IRC is quite useful, anyone who has put a little effort into it remains connected at all times. For myself, this is by keeping my computer on, but it shouldn't be too hard to find a BNC service (possibly Quassel), so you don't Ali-G people.

Also, there is no such thing as "what the main server will be like in 2 weeks time".

I oppose special privileges for testers, or a closed testing team. For "secret" events we still have the Council forum.
Former programmer for the TMWA server.
User avatar
Wombat
TMW Adviser
TMW Adviser
Posts: 1532
Joined: 08 Aug 2008, 16:31

Re: Forming a beta testing team

Post by Wombat »

Derpella has got in contact on IRC and has begun discussing testing with Jenalya. Nard is more than welcome to attempt the same. Practicing a beta testing team is probably the only way to do it. There isn't anything new needed to make this happen. It is a group of testers working on the testing servers we offer that communicate with each other, the development team and beta testing administration about what is going on. Using IRC and the forums offered here, just about any issue can be discussed.
Current character is "Abolish".
Frost
TMW Adviser
TMW Adviser
Posts: 851
Joined: 09 Sep 2010, 06:20
Location: California, USA

Re: Forming a beta testing team

Post by Frost »

This is an excellent discussion. I'm also really glad that more people are interested in testing.

All content changes are publicly available on gitorious*. Technically, this means there are no "secret changes." On the other hand, some changes consist of many small details which are not only hard to summarize, but can even be hard to keep in your head at one time.
An example of this is something like changing spelling of item names to make them consistent across all files. A simple concept (clean up names) becomes tedious in execution because literally hundreds of lines in multiple files have small changes -- and any change has potential for errors like changing a towel into a piece of fruit.
In such a situation, "git diff" provides the authoritative list of changes.

There is a testing server at caliban.homeip.net. The login message describes what content needs to be tested.
This server has two specific features: First, it is under the control of the content developers, so they can make changes, restart, or do whatever will assist or accelerate testing. Second, it is available for anyone to test on; even casual players are welcome to help.
Derpella, of course you're welcome to set up a private testing server. The source code and content are freely available from git. If you have problems or questions, just ask on IRC or in the forums.

Please remember that everyone involved in this conversation -- player, developer, admin, GM, or poet -- cares about the game and wants to see it improve. I happen to read the forums and watch IRC, and most of the conflict I've seen in this thread comes from misunderstandings and not from different goals or antipathy towards other contributors.


*Specific events are sometimes withheld from git until the last minute to prevent spoilers. Testing for these is coordinated closely with content developers.
You earn respect by how you live, not by what you demand.
-unknown
User avatar
Nard
Knight
Knight
Posts: 1113
Joined: 27 Jun 2010, 12:45
Location: France, near Paris

Re: Forming a beta testing team

Post by Nard »

I suggest the following then:
A testing group is constituted: Derpella and Okurweniec (and me if needed) are the first members. None of us ask for any kind of privilege, we are just ordinary players.
We will tell you the permanent members names when they confirm their will.
Tasks and places to report will be given by Jenalya (or you Wombat?) on the development summary basis.
Feedback and Player talk forums sometimes seem a good starting choice (to me :) ).
We 3 can use mantis and manasource bug reporting tool and report on them but I suggest we do it after the bug status is confirmed and maybe it's origin (client, content ...) if possible.
(I hear and undertstand you o11c but I think a bugtracker is the good tool and it is still used :/ . However the decision is up to you and GHP when you have taken one)
In Emergency cases (critical bugs, which involve server freeze for example) we also report on IRC,dev channel and highlight the topic to Jenalya and Wombat and... :?:
I will propose to the discussion in this thread two texts (bug reporting and testing) that could be included in announcements and in "how to help"
I will also submit corresponding topics to wiki when the text is agreed, at the moment, the bug reporting link leads to mantis.

Concerning testing procedure. : I will not read any script before it is released and tested (thus I may miss economical error changes). I will stop reading content development forums to keep my objectivity as high as possible. I will test first with a char with a similar satus (same) as in official server except if I am asked to proceed differently. I will create a bunch of chars with various status to have some kind of repeatability ...
Last edited by Nard on 09 Dec 2011, 09:08, edited 1 time in total.
"The language of everyday life is clogged with sentiment, and the science of human nature has not advanced so far that we can describe individual sentiment in a clear way." Lancelot Hogben, Mathematics for the Million.
“There are two motives for reading a book; one, that you enjoy it; the other, that you can boast about it.” Bertrand Russell, Conquest of Happiness.
"If you optimize everything, you will always be unhappy." Donald Knuth.
User avatar
Derpella
Novice
Novice
Posts: 198
Joined: 14 Jun 2011, 08:08
Location: Poland

Re: Forming a beta testing team

Post by Derpella »

I like o11c's idea of making a tiny subforum for betatesters. Every change and every issue would have their own topic.
Yes, unfortunately sitting on IRC 24/7 is impossible for me, especially when I am getting disconnected very often.
I'm Image too much.
User avatar
Nard
Knight
Knight
Posts: 1113
Joined: 27 Jun 2010, 12:45
Location: France, near Paris

Re: Forming a beta testing team

Post by Nard »

Frost wrote:[...] most of the conflict I've seen in this thread comes from misunderstandings and not from different goals or antipathy towards other contributors.
to be perfectly clear: I have no a priori antipathy towards anybody. I have disagreements, yes. My only request is to be a priori considered as sincere, I am not a troller. If my tone seems aggressive to someone, I do apologize, this was and is never my first intention. And if I complained about some persons attitude it is because, as everybody, I don't like being discredited, and because I don't like the image of the development team that result from it. It is obvious that misunderstanding is the main cause of conflict here as in IRL; I will try to better take it into account.

http://forums.themanaworld.org/viewtopi ... =4&t=12325

Nard
"The language of everyday life is clogged with sentiment, and the science of human nature has not advanced so far that we can describe individual sentiment in a clear way." Lancelot Hogben, Mathematics for the Million.
“There are two motives for reading a book; one, that you enjoy it; the other, that you can boast about it.” Bertrand Russell, Conquest of Happiness.
"If you optimize everything, you will always be unhappy." Donald Knuth.
User avatar
Nard
Knight
Knight
Posts: 1113
Joined: 27 Jun 2010, 12:45
Location: France, near Paris

Re: Forming a beta testing team

Post by Nard »

Frost wrote:There is a testing server at caliban.homeip.net. The login message describes what content needs to be tested.
I don't think the login message is sufficient to include all the things to be tested, though it is necessary for an occasional tester.

the port 6902 remains a problem to me. I haven't time (and will too) enough to level a char up to 90 and even if I had, it took me about a year to reach 95 on official server with only one restat. I could reduce it to maybe 6 months with manaplus auto-follow and multiboxing (2 chars) :mrgreen:, maybe shorter if I had one of these bot clients (I heard a well known botting girl speak about a mean of one level per day until 90).
I suggest, if possible, that both caliban servers share the same user database except the GM status. Then the second server could have more testing interest.
"The language of everyday life is clogged with sentiment, and the science of human nature has not advanced so far that we can describe individual sentiment in a clear way." Lancelot Hogben, Mathematics for the Million.
“There are two motives for reading a book; one, that you enjoy it; the other, that you can boast about it.” Bertrand Russell, Conquest of Happiness.
"If you optimize everything, you will always be unhappy." Donald Knuth.
User avatar
Crush
TMW Adviser
TMW Adviser
Posts: 8046
Joined: 25 Aug 2005, 16:08
Location: Germany

Re: Forming a beta testing team

Post by Crush »

There is no need to level a character yourself. The testing server allows you to make yourself a GM, which allows you to cheat your character and equipment to any level you need for efficient testing.
Jenalya on http://forums.themanaworld.org/viewtopic.php?f=13&t=13388 wrote:After creating a new account you can type "@gm gm" to get gm level 60 and additionally the command "@item". A list of commands can be found here:
http://wiki.themanaworld.org/index.php/GM_Commands
  • former Manasource Programmer
  • former TMW Pixel artist
  • NOT a game master

Please do not send me any inquiries regarding player accounts on TMW.


You might have heard a certain rumor about me. This rumor is completely false. You might also have heard the other rumor about me. This rumor is 100% accurate.
User avatar
Nard
Knight
Knight
Posts: 1113
Joined: 27 Jun 2010, 12:45
Location: France, near Paris

Re: Forming a beta testing team

Post by Nard »

Crush wrote:As far as I remember, the testing server allows you to make yourself a level 80 GM, which allows you to cheat your character and equipment to any level you need for efficient testing.
No Crush, there are two testing servers, the one you refer to is on port 6901 and allows you to be GM 60 with some extensions (@item), the one on port: 6902 doesn't allow more possibilities than the official one .
http://forums.themanaworld.org/viewtopi ... 13&t=13388
"The language of everyday life is clogged with sentiment, and the science of human nature has not advanced so far that we can describe individual sentiment in a clear way." Lancelot Hogben, Mathematics for the Million.
“There are two motives for reading a book; one, that you enjoy it; the other, that you can boast about it.” Bertrand Russell, Conquest of Happiness.
"If you optimize everything, you will always be unhappy." Donald Knuth.
User avatar
Nard
Knight
Knight
Posts: 1113
Joined: 27 Jun 2010, 12:45
Location: France, near Paris

Re: Forming a beta testing team

Post by Nard »

About the GM commands:
I noticed as a GM on auldsbel server that some @ commands don't work as they are described in both wiki (http://wiki.themanaworld.org/index.php/GM_Commands) and @help file. Who is best qualified for a report?
"The language of everyday life is clogged with sentiment, and the science of human nature has not advanced so far that we can describe individual sentiment in a clear way." Lancelot Hogben, Mathematics for the Million.
“There are two motives for reading a book; one, that you enjoy it; the other, that you can boast about it.” Bertrand Russell, Conquest of Happiness.
"If you optimize everything, you will always be unhappy." Donald Knuth.
User avatar
Freeyorp101
Archivist Prime
Archivist Prime
Posts: 765
Joined: 04 Nov 2008, 09:17
Location: New Zealand

Re: Forming a beta testing team

Post by Freeyorp101 »

Nard wrote:About the GM commands:
I noticed as a GM on auldsbel server that some @ commands don't work as they are described in both wiki (http://wiki.themanaworld.org/index.php/GM_Commands) and @help file. Who is best qualified for a report?
Which commands, and how so?

Generally speaking, it is better (and faster) to just explain the problem than to ask for someone to explain the problem to.

Also, other servers may have different access level configurations than ours.


---Freeyorp
(09:58:17) < tux9th> Freeyorp: your sig on the forums is kind of outdated
User avatar
Crush
TMW Adviser
TMW Adviser
Posts: 8046
Joined: 25 Aug 2005, 16:08
Location: Germany

Re: Forming a beta testing team

Post by Crush »

Errors on the wiki should be fixed by the one who notices them - that's the point of a wiki. When the documentation is wrong, you should make it correct. When the command is buggy, the documentaton should say so until the bug is fixed (something like "This command is supposed to do [expected behavior], but note that as of [current date] there is a bug which causes it to do [unexpected behavior] instead when [condition] - see [link to mantis issue] for details").

Errors in @help are a server programming issue. The most active server programmer at the moment is o11c. When you think that it's just the text which needs to be fixed, you should report it to him directly. When you think it would be better to change the command to behave like the documentation says it does, it very likely requires a more complex bugfix, and you should create an issue on http://mantis.themanaworld.org
  • former Manasource Programmer
  • former TMW Pixel artist
  • NOT a game master

Please do not send me any inquiries regarding player accounts on TMW.


You might have heard a certain rumor about me. This rumor is completely false. You might also have heard the other rumor about me. This rumor is 100% accurate.
Post Reply