Page 1 of 2

[REQ] Another alternative client (Difficulty 5/5 Team-Sized)

Posted: 06 Feb 2020, 01:17
by jesusalva

Role wanted: Coder
Difficulty: 5/5 Difficult
Milestone: N/A
Priority: Optional, Important

Language: C++ (QtQuick)
Origin: viewtopic.php?f=56&t=20818


First of all, we will keep supporting ManaPlus for Desktop. With this out of the way.
There's the idea of using tales-client (Source of Tales client) for Mobile. It have a simpler interface, is easier to use on mobile, etc.

To add Evol2/Hercules support with a backend library on tales-client, which would also allow us to make other clients in future and even rewrite ManaPlus in future with a more modern codebase, it is needed a lot of work. Therefore, we plan in assembling a small team of volunteers which are interested on the task.

Obviously, if everyone is fine with using ManaPlus on Mobile, there's no problem.
You can ask Ablu for help with tales-client, he is always on IRC.

Frequently Asked Questions
Q. How does Tales-Client looks like? Where can I find it?
A. http://www.sourceoftales.org/screenshots.html for screenshots
https://gitlab.com/tales/tales-client for git repository

Q. What's the reward?
A. There's already US$ 100 collected to those working on it.
That's how much money we've collected thus far, and for everyone working on it.
So more developers means this will be less money for each, and we'll need to collect more money.
But think on it as an additional incentive.

We may also later add items to rewards.

Q. How much technical knowledge I need to have?
A. You must know about networking and C++. Studying ManaPlus, TalesClient, Evol2 and Hercules source codes also help.
Git and GitLab knowledge helps with coordination.

Q. How do I volunteer myself? How will they reach me out?
A. It is better to write here of your intention to help. A Discord also allows us to contact you faster (but you can be on IRC as well).
GitLab username is also a must.

Q. Why don't devs do it themselves?
A. Lack of (wo)manpower currently skilled in C++.
Also, everyone is busy working on something else already.
And again, this is optional priority.

Q. Who is responsible for deciding upon the final version?
A. Client development is handled outside TMW's, so the final word would be Ablu's and the Source of Tales development team.

Q. I want to make my own client, not to use Tales' client.
A. This is a request for a backend, nor for a client rewrite.
Several other applications and languages can interact with it, and of course, you can always copy-paste the backend and use it elsewhere.

Q. I don't know how to code but I want to contribute on my own way.
A. Alternative ways of contributing include donating real money (which is what a certain group did, as you can see), donating ingame GP and items, etc.
But we still need volunteers, or at least people willing to learn C++ and tackle this project.
(Actually we might accept even prayers so feel free to contribute as you want! This is free software.)

Q. How long do you estimate this task to take?
A. Several weeks. Depends on number of volunteers, their coordination, and several other factors.

Q. Will this delay official server release?
A. Probably not.

Q. What packet version should the backend use?
A. We need server packet version 20170517 support.
Support for earlier packet versions is not required. Support for later packet versions is appreciated, in special, if you manage add support for all 2019 packet versions, we will have inventory boosting items (what, 100 slots isn't enough for you?! Weird.)


Anything else, feel free to contact here as well.

Re: [REQ] Another alternative client (Difficulty 5/5 Team-Sized)

Posted: 08 Feb 2020, 18:34
by tomminator
So I took a look at the tales client sourcecode and it seems like all you have to do is:
  1. change the packet numbers found in tales-client/src/mana/protocol.h to the ones found in evol-all/server-code/src/map/packets.h
  2. add missing but needed packets and their functionality to tales-client(most likely not all hercules packets are used by tmw)
  3. then also some parsing is most likely also needed, because the data in the packets may not be what tales-client expect
  4. hercules/evol uses login- char- and map-server, source of tales uses chat- account- and map-server (this might or might not be a very big issue)
  5. figure out how tales-client handles the client data and adapt it so it can handle the current client data. (we cant modify existing client data or manaplus will stop working)
  6. add missing features (i dont know if tales-client supports stuff like heights in maps and adding cards to items via inventory)
easy peasy!!! :alt-1: :alt-1: :alt-1:

Re: [REQ] Another alternative client (Difficulty 5/5 Team-Sized)

Posted: 08 Feb 2020, 21:11
by Ablu
The best way would probably to develop a lib which implements the network API and populates the models of SoT. The code is generally designed as a library (and even built as one). So one would not need to fork everything but could benefit from a shared code base.

Re: [REQ] Another alternative client (Difficulty 5/5 Team-Sized)

Posted: 09 Feb 2020, 03:14
by jesusalva
tomminator wrote: 08 Feb 2020, 18:34
  1. figure out how tales-client handles the client data and adapt it so it can handle the current client data. (we cant modify existing client data or manaplus will stop working)
  2. add missing features (i dont know if tales-client supports stuff like heights in maps and adding cards to items via inventory)
a. I believe tales-client uses an outdated version of our own client-data files. So it should mostly work. Which brings me to

b. Heights are cool. Adding cards to items via inventory is cool (and important, on ML, the craft system is almost 100% based on drag'n'drop). However, these are only critical if we were trying to replace ManaPlus entirely.

What we want is an alternative, which is easy to play on mobile. Given we aren't focusing on the mobile-only players (yet), I think it is fine to start with only the critical gameplay features and having optional features on Desktop only.

In other words: It is still a huge task, but initially it is not so big as your comment suggests :alt-3:

Keep in mind tales-client techinically comes from Mana Source, and ManaPlus also comes from Mana Source (but more directly).

Re: [REQ] Another alternative client (Difficulty 5/5 Team-Sized)

Posted: 09 Feb 2020, 09:26
by Ablu
jesusalva wrote: 09 Feb 2020, 03:14

a. I believe tales-client uses an outdated version of our own client-data files. So it should mostly work.
The Mana Server does not differ between client and server data. The files are shared. Thus the format only shares the XML structure, but the content will differ. But the parsing logic is fairly decoupled and could be exchanged. But this is not really a thing about newer or outdated files, things are simply different. I would advise that you try to understand how stuff on the Mana Server rather than trying to bend things into the way they worked for TMW in the past.

Regards,
Ablu

Re: [REQ] Another alternative client (Difficulty 5/5 Team-Sized)

Posted: 09 Feb 2020, 15:33
by jesusalva
Ablu wrote: 09 Feb 2020, 09:26
jesusalva wrote: 09 Feb 2020, 03:14

a. I believe tales-client uses an outdated version of our own client-data files. So it should mostly work.
The Mana Server does not differ between client and server data. The files are shared. Thus the format only shares the XML structure, but the content will differ. But the parsing logic is fairly decoupled and could be exchanged. But this is not really a thing about newer or outdated files, things are simply different. I would advise that you try to understand how stuff on the Mana Server rather than trying to bend things into the way they worked for TMW in the past.

Regards,
Ablu
I probably removed from my italic text but now I'm regretting it.
What I deleted was about how after splitting from Mana Source both followed their own ways etc.,
but I feel like that argument would be wasted.
Thus I stop commenting here.

Re: [REQ] Another alternative client (Difficulty 5/5 Team-Sized)

Posted: 15 Feb 2020, 10:38
by tomminator
So lets try to get this ball rolling!

I used protocol.h as a starting point and looked at where it is used everywhere. Presuming all the network related code depends on protocol.h

These files depend on protocol.h:

/tales-client/src/mana/gameclient.cpp
/tales-client/src/mana/messagein.h
/tales-client/src/mana/chatclient.cpp
/tales-client/src/mana/accountclient.cpp
/tales-client/src/mana/messageout.h
/tales-client/src/mana/being.h

I presume if we are going to implement a lib for the network api we have to modify these files to make use of this lib

The following files have protocol.h as an #include, but as far as i can see the don't make use of it:

/tales-client/src/mana/resource/attributedb.h
/tales-client/src/mana/inventorylistmodel.h
/tales-client/src/mana/character.cpp
/tales-client/src/mana/resource/hairdb.h
/tales-client/src/mana/resource/itemdb.h
/tales-client/src/mana/resource/monsterdb.h
/tales-client/src/mana/resource/npcdb.h
/tales-client/src/mana/resource/racedb.h
/tales-client/src/mana/resource/spritedef.h

I removed #include "mana/protocol.h" from them and compiling works.

the following files depend on protocol.h for the declaration of 'ValueType':

/tales-client/src/mana/messageout.h
/tales-client/src/mana/messagein.h


maybe we can declare 'ValueType' in another header file? This would make only the following files dependent on protocol.h and perhaps needs refactoring for the network lib.

/tales-client/src/mana/gameclient.cpp
/tales-client/src/mana/chatclient.cpp
/tales-client/src/mana/accountclient.cpp
/tales-client/src/mana/being.h

Would this be a good starting point?

Re: [REQ] Another alternative client (Difficulty 5/5 Team-Sized)

Posted: 15 Feb 2020, 10:54
by Ablu
The protocol.h is mostly copied from the manaserv code base and ideally should stay that way. MessageIn and MessageOut are also very manaserv specific. What you probably want to do is to separate the network handling from the view model functionality in the *Client classes. Then the hercules lib could reimplement the clients and the models would be shared (they would not depend on the protocol any longer).

Being only depends on the protocol for things like direction and gender. That could easily be refactored, but again splitting the *Client classes into network handling and view models would be the largest work. Feel free to join #mana on freenode to discuss stuff :)

Regards,
Ablu

Re: [REQ] Another alternative client (Difficulty 5/5 Team-Sized)

Posted: 22 Feb 2020, 14:40
by tomminator
Ablu wrote: 15 Feb 2020, 10:54 What you probably want to do is to separate the network handling from the view model functionality in the *Client classes.
so for example

Code: Select all

void GameClient::authenticate(const QByteArray &token)
{
    // Send in the security token
    MessageOut msg(Protocol::PGMSG_CONNECT);
    msg.writeString(token, 32);
    send(msg);
}
would be replaced by something like: (following code is pseudocode since i don't have experience coding c++/qt yet)

Code: Select all

void GameClient::authenticate(const QByteArray &token)
{
    // Send in the security token
    if servertype == tales{
        tales::authenticate(const QByteArray &token); 
        }
    if servertype == hercules{
        hercules::authenticate(const QByteArray &token); 
        }
}
and then tales::authenticate() would be

Code: Select all

void tales::authenticate(const QByteArray &token)
{
    // Send in the security token
    MessageOut msg(Protocol::PGMSG_CONNECT);
    msg.writeString(token, 32);
    send(msg);
}
and hercules::authenticate() would be

Code: Select all

void tales::authenticate(const QByteArray &token)
{
    // Send in the security token
    // hercules specific code
}
and the tales and hercules functions would be in their seperate *.cpp? maybe even in separate directories?
Ablu wrote: 15 Feb 2020, 10:54 Feel free to join #mana on freenode to discuss stuff :)
my name on irc is toams, I like irc but for longer questions like this i think forums are better :-)

maybe its better if you separate one networking function as an example? so i can base my work on that? only the tales specific part would be enough.

Re: [REQ] Another alternative client (Difficulty 5/5 Team-Sized)

Posted: 22 Feb 2020, 19:32
by Ablu
Hi,
tomminator wrote: 22 Feb 2020, 14:40 would be replaced by something like: (following code is pseudocode since i don't have experience coding c++/qt yet)

Code: Select all

void GameClient::authenticate(const QByteArray &token)
{
    // Send in the security token
    if servertype == tales{
        tales::authenticate(const QByteArray &token); 
        }
    if servertype == hercules{
        hercules::authenticate(const QByteArray &token); 
        }
}
You would not want to add if statements for each type. Instead one would simply add a separate QML Object called "HerculesGameClient" maybe. This would be a class in a separate lib which is exported as QML module which can then be used instead of the GameClient of the mana package.

The class names can then simply be exchanged at https://gitlab.com/tales/tales-client/- ... qml#L42-87 (or the equivalent location in the potential TMW-Client code). So essentially one would use QML for dependency injection. However, currently the GameClient also handles logic stuff which is network independent. So that would need to be refactored out of the class to a more generic place.
tomminator wrote: 22 Feb 2020, 14:40 maybe its better if you separate one networking function as an example? so i can base my work on that? only the tales specific part would be enough.
As I said, I think it is more about splitting the non-network related stuff out of the *Client classes. There would need to be a defined interface to the game logic which is not depending on the actual protocol. If you do not fully understand what I am saying it may be the simplest solution if you try to implement the Hercules networking by simply replacing the existing *Client classes (without changing the interface to other classes! (so only add new classes for utilities, but do not do major changes to any other files). That would be a "quick" proof of concept and would be easy to turn into the library format that I am intending :)

Regards,
Ablu

Re: [REQ] Another alternative client (Difficulty 5/5 Team-Sized)

Posted: 24 Jul 2020, 02:03
by Aeuda

Hello, toams==tomminator sent me.

I think there are 2 rather sepparate issues here.

Issue 1)
The TMW2 community(and some legacy players) wants a better mobile client.

Issue 2)
The Mana and 'Source of Tales' community wants a networking library that will make it easier to develop new clients and to share code between different client projects. (If I understand their desires correctly)

Starting with issue 2, I think the best way to acchieve that goal would be to restructure the networking code from ManaPlus, repackage it as a library with an easy to use API and then fork it as a separate project that could be used by both ManaPlus and Source of Tales. (as well as other clients that may be developed in the future) I'm not really much of a programmer, but I'd be willing to go off on a limb and say I could do that in less than a year mostly by myself with a little guidance. The networking code for ManaPlus is a bit convoluted and messy, but at least it's neatly contained and (for the most part) it has a very clear sepparation between code that deals with the network and code that responds to/sends messages. Also, since MP already has support for all (most?) of the features and such it would be easier to move code arround than to write new code.

On issue 1, While I support client diversity, I think ManaPlus already has most of the features needed to make it useable on mobile (touch interface things), they just need to be coordinated (add a "enable all mobile-friendly features" button) and fixed (for example the 'large' onscreen joystick has a functional area the same size as the medium one). The only features I can see that Source of Tails has that MP doesn't is convenient window management, larger buttons, and a better ratio between the User Interface size and game size. Guichan (the windowing library used by ManaPlus) obviously isn't great for adding these conveniences to MP, but I still think it would be faster and easier to improve MP than to get Source of Tales to work with Hercules.

Tl;DR creating a networking backend for manaclients is a good and worthwhile thing to do, but IMO it's not the best way to get a playable mobile release quickly or easily. The most sensible course of action in my view is to improve MP first then work on a networking backend for clients.

P.S This just occured to me, but you might want to keep in mind that creating an easy to use networking api would (probably) also make it easier to develop sophisticated bots run by headless clients. Just a thought.


Re: [REQ] Another alternative client (Difficulty 5/5 Team-Sized)

Posted: 24 Jul 2020, 10:07
by jesusalva

For the record, as far as I know, ManaPlus (M+) no longer builds for mobile.

I mean, since NaCl got deprecated.

Newer Android versions, thanks to Google «insert insult here» policy of changing the API in breaking non-forward-compatible ways, probably will not have ManaPlus on their store in a not-so-distant future.

Unless someone fixes it.

That is why using SoT client is better than changing M+ - even if you succeed you have not truly succeeded because the libraries which gave support have changed; Resulting in even more work.

Keep in mind "mobile-support" is not "add touch controls to it"; Mobile has its own set of guidelines and design documentation.

In this sense, SoT client is more stable Android-wise. Wait. You're forgetting iOS! The poor WIldX-eses, alone with their (trusty?) iPads; Do you really hate them so much? :D

Also, you understood #2 but for the wrong cause. (Wrong answers leading to correct conclusions case). The idea from segregating the network library is not from SoT, but from TMW. Source of Tales uses ManaServ and multiple times showed no interest in changing that.

However, it is not on TMW's best behalf of interest to limit ourselves to ManaPlus. Why? Because M+ is a different project. What would TMW do if 4144, for example, had a sudden heart attack? Well, it is probably the quickest and maybe even the easiest way to kill TMW for good.

In a contingent scenario already for Open Source, the more compatible clients you have, all the better. Other solution would have multiple people of 4144's skill and knowledge level working on M+, but that is not only improbable, but trying to do so is much more prone to failure in an open source environment. Wanna see an example? 4144 did not begin working for Mana, he started with a patch you could apply over Mana. Which later became a fork. And then ManaPlus.

Would we had M+ today if we tried to force 4144 into developing the original Mana client? I don't think so. (For reference, the original mana client is dead and probably beyond recovery.)

(Another optional challenge you can take for fun is, "try to compile M+ for Windows". I am yet to find someone else other than 4144 able to achieve that.)


Re: [REQ] Another alternative client (Difficulty 5/5 Team-Sized)

Posted: 24 Jul 2020, 11:52
by Aeuda

Ahh, It seems I had many misunderstandings :)

I thought I heard Myu. had gotten some phone client to work a month or so ago, so I presumed M+ still was accessible on mobile.
If getting M+ to mobilize is harder than I thought it would be then enabling the development of other clients does seem like the option to prioritize.

Segregating the network code from M+ and making it its own library seems hard but doable, even given my non-existent knowledge or skills. . .
Reworking SoT to then use that library and implement missing functionality is probably a bit above my skill level.

Having a separate network component could also make it easier to make TMW more independent from different servers if we could get them to support the same api (not very likely, but not 100% impossible either. . .). Hercules is great, but IMO it's a bit cluttered with Ragnarok things which are irrelevant to TMW.


Re: [REQ] Another alternative client (Difficulty 5/5 Team-Sized)

Posted: 24 Jul 2020, 13:05
by jesusalva
Aeuda wrote: 24 Jul 2020, 11:52

Reworking SoT to then use that library and implement missing functionality is probably a bit above my skill level.

Yes, but Ablu and the SoT team offered to help here :D Which is how you got #2 wrong :mrgreen:


Re: [REQ] Another alternative client (Difficulty 5/5 Team-Sized)

Posted: 24 Jul 2020, 14:16
by Aeuda

Well, sounds like a fun project. Let's see how far in I can get before I drown :P