by hsn915 on 1/16/2025, 5:50:07 AM
Are you talking about full on translation, or just translating individual words?
Yes macOS has builtin OCR, but what about the translation itself? You still need a cloud service for that.
If you want to read Japanese but lookup words/kanji you don't understand, I have a tool for that (full disclosure: it's a paid app) that I developed for exactly this purpose. Yomitai: https://yomitai.app/
For macOS users, I highly recommend trying Safari's built-in translation service. It uses Apple's Vision API for OCR and handles multi-image content incredibly quickly.
The Good: Speed: It’s noticeably faster than most plugins that rely on uploading images to a server, performing OCR, and then downloading translations. Ease of Use: Works seamlessly within the browser, with minimal setup. The Flaws: Poor Layout Handling: The recognized text doesn’t adapt well when zooming in or out in the browser. Formatting Issues for Japanese: Translated Japanese manga text often appears as awkward vertical text, rotated 90 degrees in a way that breaks immersion. The Idea: Imagine combining Apple's Vision API for lightning-fast OCR with a custom translation model tailored to manga (which likely already exists). This would address both speed and accuracy while allowing for more intelligent text placement and formatting.
Here’s the big question:
How can we enable Apple’s Vision API in other browsers like Chrome? Is it possible to integrate specialized translation models into this pipeline to enhance translation quality? Could we improve text rendering to better align with the original manga layout? Would love to hear thoughts on this. Have you experimented with similar setups, or is there another solution that’s already out there?