Merge pull request #179 from tak2hu/dev

.vscode

.VSCodeCounter

.idea

*.zip

*.ipynb

*.dll

# Features

* Fully automated translation  

  - Support automatic text-detection, recognition, removal, and translation, overall performance is dependent upon these modules.

  - lettering is based on the formatting estimation of the original text.

  - Support automatic text-detection, recognition, removal, and translation. Overall performance is dependent upon these modules.

  - Lettering is based on the formatting estimation of the original text.

  - Works decently with manga and comics.

  - Improved manga->English, English->Chinese typesetting (based on the extraction of balloon regions.).

  - Support search & replace

  - Support export/import to/from word documents

# Usage

# Installation

## Executable Binaries

Windows users can download Ballonstranslator-x.x.x-core.7z from [MEGA](https://mega.nz/folder/gmhmACoD#dkVlZ2nphOkU5-2ACb5dKw) or [Google Drive](https://drive.google.com/drive/folders/1uElIYRLNakJj-YS0Kd3r3HE-wzeEvrWd?usp=sharing)(note: you also need to download latest Ballonstranslator-1.3.xx from GitHub release and extract it to overwrite **Ballontranslator-1.3.0-core** or older installation to get the app updated.)

**Windows users** can download Ballonstranslator-x.x.x-core.7z from [MEGA](https://mega.nz/folder/gmhmACoD#dkVlZ2nphOkU5-2ACb5dKw) or [Google Drive](https://drive.google.com/drive/folders/1uElIYRLNakJj-YS0Kd3r3HE-wzeEvrWd?usp=sharing) (note: you also need to download latest Ballonstranslator-1.3.xx from GitHub release and extract it to overwrite **Ballontranslator-1.3.0-core** or older installation to get the app updated.)

## Run the source code

If you're not on Windows or may want to run the latest development.

```bash

# Clone this repo

$ pip install -r requirements.txt

```

Notes: 

- To update run `git pull`

- `git clone -b dev` for dev branch, or git `checkout dev`

Install pytorch-cuda to enable GPU acceleration if you have a NVIDIA GPU.

```bash

Download the **data** folder from [MEGA](https://mega.nz/folder/gmhmACoD#dkVlZ2nphOkU5-2ACb5dKw) or [Google Drive](https://drive.google.com/drive/folders/1uElIYRLNakJj-YS0Kd3r3HE-wzeEvrWd?usp=sharing) and move it into BallonsTranslator/ballontranslator, finally run

```bash

# For Linux or MacOS users, see [this script](https://github.com/dmMaze/BallonsTranslator/blob/master/ballontranslator/scripts/download_models.sh) and run to download ALL models

python ballontranslator

```

For Linux or MacOS users, see [this script](ballontranslator/scripts/download_models.sh) and run to download ALL models

### Apple Silicon Mac native build .app application

```

### install python 3.9.13 virtual environment

## Apple Silicon native build .app application

<details closed>

<summary>Instructions</summary>

<br>

Install python 3.9.13 virtual environment

```python

brew install pyenv mecab

env PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 3.9.13

pyenv global 3.9.13

python3 -m venv ballonstranslator

source ballonstranslator/bin/activate

```

# Clone the repository

Clone the repository

```bash

git clone https://github.com/dmMaze/BallonsTranslator.git

cd BallonsTranslator

```

# Install the dependencies

Install the dependencies

```bash

pip3 install -r requirements_macOS.txt

```

# Package the application

Package the application

```bash

cd ballontranslator

sudo pyinstaller __main__.spec

# The packaged `BallonsTranslator.app` is in the `dist` folder

# Note that the app is not functional yet, you need to go to [MEGA](https://mega.nz/folder/gmhmACoD#dkVlZ2nphOkU5-2ACb5dKw) or [Google Drive](https://drive.google.com/drive/folders/1uElIYRLNakJj-YS0Kd3r3HE-wzeEvrWd?usp=sharing), download `data` and overwrite it to `BallonsTranslator.app/Contents/Resources/data`.

# When overwriting select "``Merge``, after the overwrite is done, the application is finally packaged and complete, out of the box, just drag the application to the macOS application folder, no need to configure the Python environment again.

# Or see [this script](https://github.com/dmMaze/BallonsTranslator/blob/master/ballontranslator/scripts/download_models.sh)

```

The packaged `BallonsTranslator.app` is in the `dist` folder

Note that the app is not functional yet, you need to go to [MEGA](https://mega.nz/folder/gmhmACoD#dkVlZ2nphOkU5-2ACb5dKw) or [Google Drive](https://drive.google.com/drive/folders/1uElIYRLNakJj-YS0Kd3r3HE-wzeEvrWd?usp=sharing), download `data` and overwrite it to `BallonsTranslator.app/Contents/Resources/data`.

When overwriting select "``Merge``, after the overwrite is done, the application is finally packaged and complete, out of the box, just drag the application to the macOS application folder, no need to configure the Python environment again.

Or see [this script](ballontranslator/scripts/download_models.sh)

</details>

To use Sugoi translator(Japanese-English only), download [offline model](https://drive.google.com/drive/folders/1KnDlfUM9zbnYFTo6iCbnBaBKabXfnVJm), move "sugoi_translator" into the BallonsTranslator/ballontranslator/data/models.  

## Fully automated translation

**It is recommended to run the program in a terminal in case it crashed and left no information, see the following gif.**, Please select the desired translator and set the source and target languages the first time you run the application. Open a folder containing images that need translation, click the "Run" button and wait for the process to complete.  

# Usage

**It is recommended to run the program in a terminal in case it crashed and left no information, see the following gif.**

<img src="doc/src/run.gif">  

- The first time you run the application, please select the translator and set the source and target languages by clicking the settings icon.

- Open a folder containing images of a comic (manga/manhua/manhwa) that need translation by clicking the folder icon.

- Click the `Run` button and wait for the process to complete.

The font formats such as font size and color are determined by the program automatically in this process, you can predetermine those formats by change corresponding options from "decide by program" to "use global setting" in the config panel->Lettering. (global settings are those formats shown by the right font format panel when you are not editing any textblock in the scene)

## Image editing

## Image Editing

### inpaint tool

### Inpaint Tool

<img src="doc/src/imgedit_inpaint.gif">

<p align = "center">

Image editing mode, inpainting tool

Image Editing Mode, Inpainting Tool

</p>

### rect tool

<img src="doc/src/rect_tool.gif">

<p align = "center">

rect tool

Rect Tool

</p>

To 'erase' unwanted inpainted results, use the inpainting tool or rect tool with your **right button** pressed.  

## Text editing

<img src="doc/src/textedit.gif">

<p align = "center">

Text editing mode

Text Editing Mode

</p>

<img src="doc/src/multisel_autolayout.gif" div align=center>

<p align=center>

batch text formatting & auto layout

Batch Text Formatting & Auto Layout

</p>

<img src="doc/src/ocrselected.gif" div align=center>

<p align=center>

ocr & translate selected area

OCR & Translate Selected Area

</p>

## Shortcuts

* In the image editing mode, use the slider on the right bottom to control the original image transparency.

* The "OCR" and "A" button in the bottom toolbar controls whether to enable OCR and translation, if you disable them, the program will only do the text detection and removal.  

* Set parameters of automatic modules in the config panel.  

* ```Ctrl++```/```Ctrl+-``` to resize image

* ```Ctrl++```/```Ctrl+-``` (Also ```Ctrl+Shift+=```) to resize image.

* ```Ctrl+G```/```Ctrl+F``` to search globally/in current page.

<img src="doc/src/configpanel.png">  

- Patreon: <https://www.patreon.com/voilelabs>

- 爱发电: <https://afdian.net/@voilelabs>  

Sugoi translator is created by [mingshiba](https://www.patreon.com/mingshiba).

[Sugoi translator](https://sugoitranslator.com/) is created by [mingshiba](https://www.patreon.com/mingshiba).

## Text detection

Support English and Japanese text detection, training code and more details can be found at [comic-text-detector](https://github.com/dmMaze/comic-text-detector)

 * [manga_ocr](https://github.com/kha-white/manga-ocr) is from [kha-white](https://github.com/kha-white), text recognition for Japanese, with the main focus being Japanese manga.

## Inpainting

  * AOT is from manga-image-translator.

  * PatchMatch is an algorithm from [PyPatchMatch](https://github.com/vacancy/PyPatchMatch), this program use a [modified version](https://github.com/dmMaze/PyPatchMatchInpaint) by me. (Adobe uses this algorithm)

  * AOT is from [manga-image-translator](https://github.com/zyddnys/manga-image-translator).

  * [LaMa: Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://github.com/advimman/lama)

  * PatchMatch is an algorithm from [PyPatchMatch](https://github.com/vacancy/PyPatchMatch), this program uses a [modified version](https://github.com/dmMaze/PyPatchMatchInpaint) by me. 

## Translators

 * <s> Please change the goolge translator url from *.cn to *.com if you are not blocked by GFW. </s> Google shuts down translate service in China, please set corresponding 'url' in config panel to *.com.

 * Caiyun translator need to require a [token](https://dashboard.caiyunapp.com/).

 * Papago.

Available translators: Google, DeepL, ChatGPT, Sugoi, Caiyun, Baidu. Papago, and Yandex.

 * Google shuts down translate service in China, please set corresponding 'url' in config panel to *.com.

 * [Caiyun](https://dashboard.caiyunapp.com/), [ChatGPT](https://platform.openai.com/playground), [Yandex](https://yandex.com/dev/translate/), [Baidu](http://developers.baidu.com/), and [DeepL](https://www.deepl.com/docs-api/api-access) translators needs to require a token or api key.

 * DeepL & Sugoi translator (and it's CT2 Translation conversion) thanks to [Snowad14](https://github.com/Snowad14).

 * Sugoi translates Japanese to English completely offline.

 To add a new translator, please reference [how_to_add_new_translator](doc/how_to_add_new_translator.md), it is simple as subclass a BaseClass and implementing two interfaces, then you can use it in the application, you are welcome to contribute to the project.  

## Misc

## FAQ & Misc

* If your computer has an Nvidia GPU, the program will enable cuda acceleration for all models by default, which requires around 6G GPU memory, you can turn down the inpaint_size in the config panel to avoid OOM. 

* Thanks to [bropines](https://github.com/bropines) for the Russian localisation.  

* Add support for [saladict](https://saladict.crimx.com) (*All-in-one professional pop-up dictionary and page translator*) in the mini menu on text selection. [Installation guide](doc/saladict.md)

* Accelarate performance if you have a [NVIDIA's CUDA](https://pytorch.org/docs/stable/notes/cuda.html) or [AMD's ROCm](https://pytorch.org/docs/stable/notes/hip.html) device as most modules uses [PyTorch](https://pytorch.org/get-started/locally/).

* Fonts are from your system's fonts.

* Thanks to [bropines](https://github.com/bropines) for the Russian localization.

 No newline at end of file

  - Mendukung pencarian & penggantian kata

  - Mendukung ekspor/impor ke/dari dokumen word

# Usage

# Instalasi

Pengguna Windows dapat unduh Ballonstranslator-x.x.x-core.7z di [MEGA](https://mega.nz/folder/gmhmACoD#dkVlZ2nphOkU5-2ACb5dKw) atau [Google Drive](https://drive.google.com/drive/folders/1uElIYRLNakJj-YS0Kd3r3HE-wzeEvrWd?usp=sharing)(catatan: Anda juga perlu mengunduh Ballonstranslator-1.3.xx terbaru di rilis GitHub mengekstraknya untuk menimpa **Ballontranslator-1.3.0-core** atau instalasi yang lebih lama agar aplikasi dapat diperbarui.)

**Pengguna Windows** dapat unduh Ballonstranslator-x.x.x-core.7z di [MEGA](https://mega.nz/folder/gmhmACoD#dkVlZ2nphOkU5-2ACb5dKw) atau [Google Drive](https://drive.google.com/drive/folders/1uElIYRLNakJj-YS0Kd3r3HE-wzeEvrWd?usp=sharing) (catatan: Anda juga perlu mengunduh Ballonstranslator-1.3.xx terbaru di rilis GitHub mengekstraknya untuk menimpa **Ballontranslator-1.3.0-core** atau instalasi yang lebih lama agar aplikasi dapat diperbarui.)

## Jalankan kode sumber

Unduhlah folder **data** dari [MEGA](https://mega.nz/folder/gmhmACoD#dkVlZ2nphOkU5-2ACb5dKw) atau [Google Drive](https://drive.google.com/drive/folders/1uElIYRLNakJj-YS0Kd3r3HE-wzeEvrWd?usp=sharing) dan pindahkan ke dalam BallonsTranslator/ballontranslator, akhirnya jalankan

```bash

# Untuk pengguna Linux atau MacOS, lihat [skrip ini] (https://github.com/dmMaze/BallonsTranslator/blob/master/ballontranslator/scripts/download_models.sh) dan jalankan untuk mengunduh SEMUA model

python ballontranslator

```

Untuk pengguna Linux atau MacOS, lihat [script ini](ballontranslator/scripts/download_models.sh) dan jalankan untuk mengunduh semua model

Untuk menggunakan Sugoi translator (hanya bahasa Jepang-Inggris), unduh [offline model](https://drive.google.com/drive/folders/1KnDlfUM9zbnYFTo6iCbnBaBKabXfnVJm), pindahkan "sugoi_translator" ke dalam BallonsTranslator/ballontranslator/data/models.

## Penerjemahan sepenuhnya otomatis

**Disarankan untuk menjalankan program di terminal jika program ini mendadak berhenti dan tidak meninggalkan informasi, lihat gif berikut ini**, Pilih penerjemah yang diinginkan dan atur bahasa sumber dan target saat pertama kali menjalankan aplikasi. Buka folder yang berisi gambar-gambar yang mau diterjemahkan, klik tombol "Run" dan tunggu hingga proses selesai.  

# Penggunaan

**Disarankan untuk menjalankan program di terminal jika program ini crash dan tidak meninggalkan informasi, lihat gif berikut ini**

<img src="doc/src/run.gif">  

- Pilih penerjemah yang diinginkan dan atur sumber dan target bahasa. 

 - Buka folder yang berisi gambar manga/manhua/webtoon yang ingin diterjemahkan.

 - Klik tombol "Run" dan tunggu hingga proses selesai.

Format font seperti ukuran font dan warna ditentukan oleh program secara otomatis dalam proses ini, Anda dapat menentukan format tersebut sebelum memulai proses dengan mengubah opsi yang sesuai dari "decide by program" menjadi "use global setting" di panel konfigurasi->Lettering. (pengaturan global adalah format yang ditampilkan oleh panel format font yang tepat ketika Anda tidak mengedit blok teks apa pun di adegan)

## Image editing

#!/usr/bin/env bash

CTD_MODEL_LINK="https://github.com/zyddnys/manga-image-translator/releases/download/beta-0.3/comictextdetector.pt"

CTD_ONNX_MODEL_LINK="https://github.com/zyddnys/manga-image-translator/releases/download/beta-0.3/comictextdetector.pt.onnx"

AOT_INPAINTER_MODEL_LINK="https://github.com/zyddnys/manga-image-translator/releases/download/beta-0.3/inpainting.ckpt"

LAMA_MPE_INPAINTER_MODEL_LINK="https://github.com/zyddnys/manga-image-translator/releases/download/beta-0.3/inpainting_lama_mpe.ckpt"

SUGOI_TRANSLATOR_MODEL_LINK="https://github.com/zyddnys/manga-image-translator/releases/download/beta-0.3/sugoi-models.zip"

MANGA_OCR_MODEL_LINK="https://huggingface.co/kha-white/manga-ocr-base"

MIT48PX_OCR_MODEL_LINK="https://github.com/zyddnys/manga-image-translator/releases/download/beta-0.3/ocr-ctc.zip"

# Get the most of the models https://github.com/zyddnys/manga-image-translator/releases/tag/beta-0.3 here

# Place them in data/models

pushd $(dirname "$0") &> /dev/null

set -e 

PWD="$(pwd)"

MODELS_DIR="$PWD/../data/models"

LIBS_DIR="$PWD/../data/libs"

MODELS_DIR="$PWD/../../data/models"

LIBS_DIR="$PWD/../../data/libs"

echo $PWD

echo $MODELS_DIR

echo $LIBS_DIR

mkdir -p $MODELS_DIR

cd $MODELS_DIR

wget -c $CTD_MODEL_LINK

# Comic Text Detector

wget -c "https://github.com/zyddnys/manga-image-translator/releases/download/beta-0.3/comictextdetector.pt"

wget -c $CTD_ONNX_MODEL_LINK

# Comic Text Detector for CPU

wget -c "https://github.com/zyddnys/manga-image-translator/releases/download/beta-0.3/comictextdetector.pt.onnx"

wget -c $AOT_INPAINTER_MODEL_LINK -O aot_inpainter.ckpt

# AOT Inpainter

wget -c "https://github.com/zyddnys/manga-image-translator/releases/download/beta-0.3/inpainting.ckpt" -O aot_inpainter.ckpt

wget -c $LAMA_MPE_INPAINTER_MODEL_LINK -O lama_mpe.ckpt

# LaMa Inpainter

wget -c "https://github.com/zyddnys/manga-image-translator/releases/download/beta-0.3/inpainting_lama_mpe.ckpt" -O lama_mpe.ckpt

wget -c $SUGOI_TRANSLATOR_MODEL_LINK ; unzip -d sugoi_translator sugoi-models.zip

# Sugoi Translator

wget -c "https://github.com/zyddnys/manga-image-translator/releases/download/beta-0.3/sugoi-models.zip" ; unzip -d sugoi_translator sugoi-models.zip

wget -c $MIT48PX_OCR_MODEL_LINK; unzip ocr-ctc.zip; mv ocr-ctc.ckpt mit48pxctc_ocr.ckpt; rm alphabet-all-v5.txt

# MIT_48PX_CTC OCR

wget -c "https://github.com/zyddnys/manga-image-translator/releases/download/beta-0.3/ocr-ctc.zip"; unzip ocr-ctc.zip; mv ocr-ctc.ckpt mit48pxctc_ocr.ckpt; rm alphabet-all-v5.txt

git lfs install; git clone $MANGA_OCR_MODEL_LINK

# Manga OCR

git lfs install; git clone "https://huggingface.co/kha-white/manga-ocr-base"

mkdir -p $LIBS_DIR

echo $LIBS_DIR

    'Tiếng Việt': '',

    'čeština': '',

    'Nederlands': '',

    'français': '',

    'Français': '',

    'Deutsch': '',

    'magyar nyelv': '',

    'italiano': '',

    'polski': '',

    'português': '',

    'Italiano': '',

    'Polski': '',

    'Português': '',

    'limba română': '',

    'русский язык': '',

    'español': '',

    'Español': '',

    'Türk dili': '',

    'украї́нська мо́ва': '',  

}