Development - #182
Merged
Merged
Conversation
…merging for translation dictionary
…ion, a margin to change the partial result, fixed a bug for CacheContainerNative.
…e memory leaks in Translator
…rrectly in the initial prompt for HY-MT model
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is for beam search improvements and the integration of Translation Dictionaries for simple word translation for the 3.0 version of RTranslator, in general:
Translation Dictionaries containing data from Wiktionary, Apertium, and FreeDict have been integrated in a SQLite db (with this script), using that, when a language couple is selected, the data structures that contain all the translations for that 2 languages, to and from English, are loaded in RAM (with a consumption of around 1MB). With this data structure, the app can search a word instantly and obtain its eventual translations. If the translation has been done with a Dictionary, the writing "Dictionary" will appear in the GUI (at least in the text translation mode), and all the synonyms are shown under the translated word.
If you want to test this version of the app early, first of all, consider that this is a development preview, so it could be pretty unstable, and it will require initial setup. To try it, you need to:
Download from here the Mozilla.zip,
Madlad.zip, HY-MT.zip, Tatoeba.zip, TranslationDictionaries.zip.
After the download, extract these folders (their name should remain "Mozilla", "Madlad", "HY-MT", "Tatoeba" and "TranslationDictionaries", with the content of these .zip directly inside the corresponding extracted folders, if you change the structure of these folders, the app will not work).
After that, create a folder named "models", inside it create a folder named "Translation", and inside it paste all the extracted folders, then insert the folder into the internal memory of the phone (in the top level).
Now download this apk, install it, open the app, enable the requested file access, and start the download.
After the download has finished, exit the app and enable all its permissions from the Android Settings (Settings -> Applications -> RTranslator).
Then re-open the app and everything should work.
By default, the models used for translation are the Mozilla ones, to select the other supported model you can select one from RTranslator's settings, at the bottom (note that probably, to execute Madlad and HY-MT, you will need a phone with at least 12GB of RAM; if the RAM won't be enough the app will crash until you wipe its data).
Also, by default, the translation with Tatoeba and Dictionaries are disabled; if you want to test those, you need to enable their use in the RTranslator settings page.