Contextual NPC Translation System

Content and general development discussion, including quest scripts and server code. TMW Classic is a project comprising the Legacy tmwAthena server & the designated improved engine server based on evolHercules.


Forum rules

This forum houses many years of development, tracing back to some of the earliest posts that exist on the board.

Its current use is for the continued development of the server and game it has always served: TMW Classic.

Post Reply
User avatar
Vasily
TMW Adviser
TMW Adviser
Posts: 15
Joined: 11 Nov 2013, 18:03
Location: Saint Petersburg

Contextual NPC Translation System

Post by Vasily »

Hello, Dear Community!

Due to some recent encounters, I decided to present the system I've been working on last year that's purpose is to make translator's life easier and the translations itself more clear and consistent. The idea was to consider the whole NPC dialogue at once, to show relations between phrases (player's phrases as well as NPC's). I've called it 'contrast', an abbreviation for 'Contextual NPC Translation System', also it should make contrast in the translation process :alt-9:

If you don't want to read such a long post, you can start exploring the results here: http://makarov.droppages.com/index.html

There was a discussion how that could be done, and there was many ideas. We've came up with thought that the solution could be a tree representation of the dialogue, where nodes represent separate phrases and links between them says that one phrase follow the another. In theory, construction of such a tree may be quite a hard task: one can come up with a script that barely will be understood by a parser in terms of phrase sequence. But, as it turns out (and as it should be, I think), most scripts have understandable structure and such a tree can be constructed without big problems.

Enough speaking. Let me show you the results I have. I attach here an image of a tree constructed by parsing Silvio dialogue. If you click on the node, a corresponding dialogue phrase shows up in a bubble (I'm sorry if it doesn't work in some browsers), also it appears on the translation panel to the right. Yellow nodes correspond to player's phrases, green to NPC's ones, blue for all other. Big black START node is an entry point for the dialogue, it starts from this node (there are some cases when it does not, but that's not very important). Multiple links going out the same node may correspond to menus, gender-related phrases or simply conditional phrases. By following the links from the START node, one should understand what's the dialogue all about.

There are function-style dialogues. One of the best examples is Julia's one. Note that the CALL nodes stand for function calls. If you hold your mouse clicked on them, corresponding function entry nodes are highlighted. If you right-click on a call node, you will be centered on the entry point of the function the call is made to. RET nodes stands for leaving the function, in most cases returning to the place where the call was made.

The panel to the right is made specially for translators. Translator gets her Transifex API key here https://www.transifex.com/user/settings/api/ and puts it in the corresponding field on the panel, same for the language slug (it is 'ru' for me) she translates into. Then she loads all strings by pressing 'Get strings' button and waits for a message about successful loading. Translated phrases' nodes will be bounded green, untranslated remain red. Translation itself is done by entering translation string into the second big field on the panel and pressing 'Update' button.

All parsed scripts are shown on the page mentioned at the start and at the end of this post. Corresponding trees are accessible through hyperlinks. Links to the original scripts on the github are near. Please note that the project is in its very alpha stage, and now some scripts are not parsed, for example, because multiple NPC dialogues in one script file are not handled properly. Please note that the trees might not correctly show some dialogue cases. The trees are like an approximate image of what's happening inside the dialogue, not precise. Please be careful.

The whole project is stored inhere: https://gitlab.com/vasily_makarov/contrast. The web pages I provided the links to are generated by special environment, it is inside the repo too. HTML pages themselves are stored in my Dropbox Pages App, as you may find out from the link. Please note that if some scripts are updated, I need to update these pages too, the process is not automatical. At the moment they're all actual.

I think I've told you all I want about the contrast. I wanted to share this to hear your opinion about it and, perhaps, gain some motivation: when someone's interested, I gain a lot of motivation. I would appreciate any thoughts about it, both negative and positive. I would love to hear new ideas about it as well.

I remind you that you can explore the parsing results here:http://makarov.droppages.com/index.html. By the way, some trees are just single big START node. That mostly means they're not NPC dialogues or just does not use translation functions :alt-3:


Yours,
Vasily
Attachments
Silvio dialogue tree
Silvio dialogue tree
contrast-silvio.png (35.83 KiB) Viewed 4491 times
Last edited by Vasily on 28 Jan 2018, 22:36, edited 1 time in total.
User avatar
EJlol
TMW Furniture
TMW Furniture
Posts: 1224
Joined: 06 Sep 2005, 08:42

Re: Contextual NPC Translation System

Post by EJlol »

Looks quite nice. One thing I have to say though that the call node is not obvious where it leads to. Of course you can right click it, and see where the screen scroll to, but then you have to scroll back to the dialogue yourself. Would it be possible to show a line on hover instead? That way the screen won't become cluttered, but you are able to see where the call will lead to.

Another thing is that it is not always very clear where a line leads to, as there are multiple lines going over each other. Adding a hover event on the node, so that all the lines of that node will be on top / with a different color would be great. A different idea is to also show the parent/child nodes text in the right panel. This would be useful to get an overview for the large menu's like the main menu in Julia.

The last usability tip I want to give is that it would be nice to add somewhere which file I'm editting. Maybe you can add this in the upperleft corner under 'constrast'?

What I really like about it is that the nodes give a good overview of the dialogue structure.
User avatar
Micksha
Lead Developer (SoM)
Lead Developer (SoM)
Posts: 225
Joined: 18 Dec 2015, 15:34

Re: Contextual NPC Translation System

Post by Micksha »

Just did a quick test, on Firefox 58.0 @ Mint 18.3, just using one of the links brings me directly to it, also connection with transifex works perfect. From now it looks very great, finally we can make translations with context! This will make translating much less annoying.

I wonder about the synthax issues, like gumi just told me it should be changed (gumi> like "foo @@ bar" vs "foo %s bar"). I could not find scentences with that kind of synthax, but at least it seems the gendered tranlations do not show up. Also I am not sure if the gendered scentences should get separate nodes, or if it would be better to just display the used #0 / #1 in the same node.

I will keep an eye on your work, and I hope we will get much more dialogues ingame soon which could be full of language-linked easter eggs :)

Thanks,
- Mick
User avatar
Vasily
TMW Adviser
TMW Adviser
Posts: 15
Joined: 11 Nov 2013, 18:03
Location: Saint Petersburg

Re: Contextual NPC Translation System

Post by Vasily »

Hello!

Thank you guys for sharing your opinion and giving a useful tips. That really inspires me! So here's brief report of some of the recent changes.

Parsing system changes:
  • Multiple scripts inside one file are now handled correctly
  • Line numbers is taken in consideration
  • If script does not contain any translated phrases, it is ignored
Glad to tell you that now it parses much more scripts, as well as single-start-node scripts are removed now. Still some scripts remain a mystery for the parser.

Interface:
  • Outgoing edges are highlighted now when mouse is over the node
  • Links to the GitLab script source (as well as to the selected phrase line number) are shown now on the right bar
  • Initial page show names of the NPC's now (since they may be multiple per file)
To be honest, interface is my weak side, but I'm trying to made at least an useful prototype. Most of the changes were made because of your tips.


Yours,
Vasily
crifxer
Peon
Peon
Posts: 14
Joined: 21 May 2011, 13:56

Re: Contextual NPC Translation System

Post by crifxer »

Hi and thanks for your system which is really great!

Maybe I do something wrong but when I try to update the translation in https://gitlab.com/evol/serverdata/blob ... ne.txt#L32 and https://gitlab.com/evol/serverdata/blob ... ne.txt#L63 nothing happens.
The same on https://gitlab.com/evol/serverdata/blob ... rs.txt#L48, https://gitlab.com/evol/serverdata/blob ... rs.txt#L59 and https://gitlab.com/evol/serverdata/blob ... rs.txt#L62.

Anyway thank you again because your tool really helps a lot!
User avatar
Vasily
TMW Adviser
TMW Adviser
Posts: 15
Joined: 11 Nov 2013, 18:03
Location: Saint Petersburg

Re: Contextual NPC Translation System

Post by Vasily »

Hello!

Just wanted to tell that the tool is now available at http://danilka.pro/contrast. It now pulls the scripts from the repo and parses them every twenty minutes, so the dialogues would always be up-to-date. I made some fixes and changes of improper parsing and string issues noted by crfixer, but there are still other issues. Some scripts still are not parseable by the tool, and I am working on it. If you note a crucial script that you need to translate and it is not in the list, please let me know.

Also some node transitions may not be correct as the tool is not very clever to understand some conditional operators and other code constructions. Please follow the code of the script if you note some inconsistency in the tree. Code and line links are located just below the script title on the right.


Yours,
Vasily
Post Reply