tanpengsccd/DeepLX
1
0
Fork 0
mirror of https://github.com/OwO-Network/DeepLX.git synced 2026-06-11 15:28:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
98918dd9f6
DeepLX/translate
History
Vincent Yang 98918dd9f6
feat(translate): migrate to oneshot endpoint (#217) 
* feat(translate): migrate to oneshot endpoint to bypass www2 anti-bot

The www2.deepl.com/jsonrpc backends behind LMT_handle_texts /
LMT_handle_jobs now sit behind aggressive WAF + per-IP throttling that
returns HTTP 429 (code 1042911 "Too many requests") within a handful of
calls from any single host — making the free path effectively unusable.

The official DeepL browser extension and iOS app skip that backend
entirely for stateless single-shot translation and POST to a separate
"oneshot" endpoint on a different host pool with its own (much looser)
rate limit. It accepts anonymous traffic with a literal
`Authorization: None` header, returns plain JSON, and supports the same
language pairs.

Switch the free path to:

  POST https://oneshot-free.www.deepl.com/v1/translate
  Authorization: None
  {"text": ["..."], "target_lang": "de", "source_lang": "en"}

Pro users continue to hit oneshot-pro.www.deepl.com with their bearer
token (the `-s` flag now carries an OAuth access token rather than the
legacy dl_session cookie).

This removes:
  - the JSON-RPC envelope (jsonrpc/method/id/params/timestamp wrapper)
  - the `i`-count timestamp trick (getICount + getTimeStamp)
  - the random-id body-spacing trick (handlerBodyMethod)
  - the whatlanggo client-side detection (oneshot detects server-side)

The DeepLXTranslationResult contract is unchanged for service handlers;
Alternatives is now always nil because the oneshot endpoint does not
return alternative translations.

Verified against /translate, /v1/translate and /v2/translate routes
end-to-end (EN/DE/ZH/JA/FR pairs, multi-sentence input, autodetect, 10x
burst) — all 200 OK on an IP that was concurrently being 429'd by www2.

* fix(translate): align oneshot request bytes with the real extension

After capturing the exact bytes the Chrome extension's service-worker
fetch() emits (via an offline echo server pointed at deeplx in place of
oneshot-free.www.deepl.com) and diffing them against what we were
sending, several distinguishable signals remained. Close them all.

Headers
-------
- Origin: chrome-extension://cofdbpoegempjloogbagkncekinflcnj
  (was https://www.deepl.com — a request from www.deepl.com itself
   never lands on the oneshot endpoint, so that origin is unusual.
   The extension ID is the canonical sender.)
- Sec-Fetch-Site: cross-site
  (was same-site — wrong; chrome-extension -> www.deepl.com IS cross-site)
- Drop Referer entirely (extension SW fetch sends none)
- Drop Pragma / Cache-Control / Upgrade-Insecure-Requests / Sec-Fetch-User
  (req.ImpersonateChrome() sets these for top-level navigation; a
   fetch() never sends them — leaving them in is a strong nav-vs-XHR tell)
- Accept-Encoding: gzip, deflate, br
  (was just gzip, Go stdlib default — Chrome 120's fetch() sends all
   three; zstd only landed as a default in Chrome 123+ so leave it off)

Body
----
- Add usage_type: "Translate" and the full app_information object
  (os/os_version/app_version/app_build/instance_id) so the JSON the
  server sees is structurally identical to what background.js IN()
  assembles. Field order in oneshotRequest matches the extension's
  object-literal order so encoding/json produces byte-identical output.
- instance_id is a v4 UUID generated once at process start and reused,
  mirroring the extension's chrome.storage-pinned ID rather than
  rotating per-request (rotation would be a far stronger signal).
- All version strings (TLS handshake, User-Agent, sec-ch-ua,
  app_information.os_version) are pinned to Chrome 120 so they tell
  one consistent story.

Transport
---------
- SetBodyBytes instead of bytes.NewReader so Content-Length is set
  (an io.Reader body forces Transfer-Encoding: chunked, which a
   fetch() with JSON.stringify body never emits)
- Once we set Accept-Encoding manually, the Go stdlib disables its
  transparent decompression and req hands us raw compressed bytes.
  Handle gzip / deflate / br by hand from Content-Encoding.
- DisableAutoReadResponse so we own the body stream end-to-end.

The Chrome 120 TLS ClientHello, HTTP/2 SETTINGS frame, pseudo-header
order and sec-ch-ua claim continue to come from ImpersonateChrome()
unchanged.

Verified end-to-end:
- Outbound bytes (against a local echo server) diff-match the
  extension's observed profile on every header and on body JSON order.
- Live oneshot-free.www.deepl.com calls: 4 language pairs OK,
  /v2/translate official-API compat OK, 10x burst 10/10 200.

* chore(deps): upgrade to latest compatible versions

Run `go get -u ./...` + `go mod tidy`. Direct upgrades:

- github.com/andybalholm/brotli   1.2.0 → 1.2.1
- github.com/tidwall/gjson         1.18.0 → 1.19.0

Indirect (notable):

- github.com/bytedance/sonic       1.15.0 → 1.15.1
- github.com/bytedance/sonic/loader 0.5.0 → 0.5.1
- github.com/bytedance/gopkg       0.1.3 → 0.1.4
- github.com/cloudwego/base64x     0.1.6 → 0.1.7
- github.com/gin-contrib/sse       1.1.0 → 1.1.1
- github.com/go-playground/validator/v10 10.30.1 → 10.30.2
- github.com/goccy/go-json         0.10.5 → 0.10.6
- github.com/klauspost/compress    1.18.4 → 1.18.6
- github.com/mattn/go-isatty       0.0.20 → 0.0.22
- github.com/pelletier/go-toml/v2  2.2.4  → 2.3.1
- golang.org/x/arch                0.24.0 → 0.27.0
- golang.org/x/crypto              0.48.0 → 0.52.0
- golang.org/x/net                 0.51.0 → 0.55.0
- golang.org/x/sys                 0.41.0 → 0.45.0
- golang.org/x/text                0.34.0 → 0.37.0

github.com/imroc/req/v3 (the HTTP client we depend on for Chrome
impersonation) is already on its latest tag v3.57.0 and pins
github.com/quic-go/quic-go to <= v0.57.x — newer quic-go removed
ConnectionTracingID/ConnectionTracingKey, which req's internal/http3
still references. That constraint also holds gin-gonic/gin at v1.11.0
and gin-contrib/cors at v1.7.6 (their later versions pull quic-go
≥ 0.58 transitively). Pin quic-go to v0.57.1 to keep the build green;
revisit when req publishes a release compatible with quic-go ≥ 0.58.

Build + live oneshot end-to-end: 4 language pairs OK, /v2/translate
official-API compat OK, 8x burst 8/8 200.

* fix(translate): seed cookie jar from www.deepl.com on first call

A real chrome-extension fetch() to oneshot-free.www.deepl.com inherits
whatever cookies the browser has on .deepl.com — at minimum
`userCountry=<iso2>` and `verifiedBot=false`, both of which the
deepl.com server sets on any page load. Our outbound bytes were
otherwise extension-identical but went out cookieless, which is a
distinguishable signal.

Wire a process-wide net/http/cookiejar onto the req.Client and trigger
a single warmup GET to https://www.deepl.com/translator on the first
translate call (sync.Once). The Set-Cookie response (userCountry,
verifiedBot) lands on .deepl.com, which the jar then automatically
echoes back on every subsequent POST to oneshot-free.www.deepl.com
(cookies set on .deepl.com match any *.deepl.com subdomain).

Verified outbound:
  Cookie: userCountry=JP; verifiedBot=false

Latency cost: first call after process start pays one extra HTTP GET
(~1s warmup); subsequent calls are unaffected (sync.Once + connection
keep-alive).

Note: we cannot replicate the _ga / _ga_<id> cookies a real user
would also carry — those are set client-side by GA's JS, which a
non-browser HTTP client can't execute. The userCountry+verifiedBot
pair already matches the "first-time visitor with JS disabled" profile,
which is the closest plausible non-browser approximation.
2026-05-22 12:04:44 +08:00
..
translate.go feat(translate): migrate to oneshot endpoint (#217) 2026-05-22 12:04:44 +08:00
types.go feat(translate): migrate to oneshot endpoint (#217) 2026-05-22 12:04:44 +08:00