mirror of
https://github.com/OwO-Network/DeepLX.git
synced 2026-06-11 15:28:50 +00:00
98918dd9f6
* feat(translate): migrate to oneshot endpoint to bypass www2 anti-bot The www2.deepl.com/jsonrpc backends behind LMT_handle_texts / LMT_handle_jobs now sit behind aggressive WAF + per-IP throttling that returns HTTP 429 (code 1042911 "Too many requests") within a handful of calls from any single host — making the free path effectively unusable. The official DeepL browser extension and iOS app skip that backend entirely for stateless single-shot translation and POST to a separate "oneshot" endpoint on a different host pool with its own (much looser) rate limit. It accepts anonymous traffic with a literal `Authorization: None` header, returns plain JSON, and supports the same language pairs. Switch the free path to: POST https://oneshot-free.www.deepl.com/v1/translate Authorization: None {"text": ["..."], "target_lang": "de", "source_lang": "en"} Pro users continue to hit oneshot-pro.www.deepl.com with their bearer token (the `-s` flag now carries an OAuth access token rather than the legacy dl_session cookie). This removes: - the JSON-RPC envelope (jsonrpc/method/id/params/timestamp wrapper) - the `i`-count timestamp trick (getICount + getTimeStamp) - the random-id body-spacing trick (handlerBodyMethod) - the whatlanggo client-side detection (oneshot detects server-side) The DeepLXTranslationResult contract is unchanged for service handlers; Alternatives is now always nil because the oneshot endpoint does not return alternative translations. Verified against /translate, /v1/translate and /v2/translate routes end-to-end (EN/DE/ZH/JA/FR pairs, multi-sentence input, autodetect, 10x burst) — all 200 OK on an IP that was concurrently being 429'd by www2. * fix(translate): align oneshot request bytes with the real extension After capturing the exact bytes the Chrome extension's service-worker fetch() emits (via an offline echo server pointed at deeplx in place of oneshot-free.www.deepl.com) and diffing them against what we were sending, several distinguishable signals remained. Close them all. Headers ------- - Origin: chrome-extension://cofdbpoegempjloogbagkncekinflcnj (was https://www.deepl.com — a request from www.deepl.com itself never lands on the oneshot endpoint, so that origin is unusual. The extension ID is the canonical sender.) - Sec-Fetch-Site: cross-site (was same-site — wrong; chrome-extension -> www.deepl.com IS cross-site) - Drop Referer entirely (extension SW fetch sends none) - Drop Pragma / Cache-Control / Upgrade-Insecure-Requests / Sec-Fetch-User (req.ImpersonateChrome() sets these for top-level navigation; a fetch() never sends them — leaving them in is a strong nav-vs-XHR tell) - Accept-Encoding: gzip, deflate, br (was just gzip, Go stdlib default — Chrome 120's fetch() sends all three; zstd only landed as a default in Chrome 123+ so leave it off) Body ---- - Add usage_type: "Translate" and the full app_information object (os/os_version/app_version/app_build/instance_id) so the JSON the server sees is structurally identical to what background.js IN() assembles. Field order in oneshotRequest matches the extension's object-literal order so encoding/json produces byte-identical output. - instance_id is a v4 UUID generated once at process start and reused, mirroring the extension's chrome.storage-pinned ID rather than rotating per-request (rotation would be a far stronger signal). - All version strings (TLS handshake, User-Agent, sec-ch-ua, app_information.os_version) are pinned to Chrome 120 so they tell one consistent story. Transport --------- - SetBodyBytes instead of bytes.NewReader so Content-Length is set (an io.Reader body forces Transfer-Encoding: chunked, which a fetch() with JSON.stringify body never emits) - Once we set Accept-Encoding manually, the Go stdlib disables its transparent decompression and req hands us raw compressed bytes. Handle gzip / deflate / br by hand from Content-Encoding. - DisableAutoReadResponse so we own the body stream end-to-end. The Chrome 120 TLS ClientHello, HTTP/2 SETTINGS frame, pseudo-header order and sec-ch-ua claim continue to come from ImpersonateChrome() unchanged. Verified end-to-end: - Outbound bytes (against a local echo server) diff-match the extension's observed profile on every header and on body JSON order. - Live oneshot-free.www.deepl.com calls: 4 language pairs OK, /v2/translate official-API compat OK, 10x burst 10/10 200. * chore(deps): upgrade to latest compatible versions Run `go get -u ./...` + `go mod tidy`. Direct upgrades: - github.com/andybalholm/brotli 1.2.0 → 1.2.1 - github.com/tidwall/gjson 1.18.0 → 1.19.0 Indirect (notable): - github.com/bytedance/sonic 1.15.0 → 1.15.1 - github.com/bytedance/sonic/loader 0.5.0 → 0.5.1 - github.com/bytedance/gopkg 0.1.3 → 0.1.4 - github.com/cloudwego/base64x 0.1.6 → 0.1.7 - github.com/gin-contrib/sse 1.1.0 → 1.1.1 - github.com/go-playground/validator/v10 10.30.1 → 10.30.2 - github.com/goccy/go-json 0.10.5 → 0.10.6 - github.com/klauspost/compress 1.18.4 → 1.18.6 - github.com/mattn/go-isatty 0.0.20 → 0.0.22 - github.com/pelletier/go-toml/v2 2.2.4 → 2.3.1 - golang.org/x/arch 0.24.0 → 0.27.0 - golang.org/x/crypto 0.48.0 → 0.52.0 - golang.org/x/net 0.51.0 → 0.55.0 - golang.org/x/sys 0.41.0 → 0.45.0 - golang.org/x/text 0.34.0 → 0.37.0 github.com/imroc/req/v3 (the HTTP client we depend on for Chrome impersonation) is already on its latest tag v3.57.0 and pins github.com/quic-go/quic-go to <= v0.57.x — newer quic-go removed ConnectionTracingID/ConnectionTracingKey, which req's internal/http3 still references. That constraint also holds gin-gonic/gin at v1.11.0 and gin-contrib/cors at v1.7.6 (their later versions pull quic-go ≥ 0.58 transitively). Pin quic-go to v0.57.1 to keep the build green; revisit when req publishes a release compatible with quic-go ≥ 0.58. Build + live oneshot end-to-end: 4 language pairs OK, /v2/translate official-API compat OK, 8x burst 8/8 200. * fix(translate): seed cookie jar from www.deepl.com on first call A real chrome-extension fetch() to oneshot-free.www.deepl.com inherits whatever cookies the browser has on .deepl.com — at minimum `userCountry=<iso2>` and `verifiedBot=false`, both of which the deepl.com server sets on any page load. Our outbound bytes were otherwise extension-identical but went out cookieless, which is a distinguishable signal. Wire a process-wide net/http/cookiejar onto the req.Client and trigger a single warmup GET to https://www.deepl.com/translator on the first translate call (sync.Once). The Set-Cookie response (userCountry, verifiedBot) lands on .deepl.com, which the jar then automatically echoes back on every subsequent POST to oneshot-free.www.deepl.com (cookies set on .deepl.com match any *.deepl.com subdomain). Verified outbound: Cookie: userCountry=JP; verifiedBot=false Latency cost: first call after process start pays one extra HTTP GET (~1s warmup); subsequent calls are unaffected (sync.Once + connection keep-alive). Note: we cannot replicate the _ga / _ga_<id> cookies a real user would also carry — those are set client-side by GA's JS, which a non-browser HTTP client can't execute. The userCountry+verifiedBot pair already matches the "first-time visitor with JS disabled" profile, which is the closest plausible non-browser approximation.
49 lines
1.9 KiB
Modula-2
49 lines
1.9 KiB
Modula-2
module github.com/OwO-Network/DeepLX
|
|
|
|
go 1.25.0
|
|
|
|
require (
|
|
github.com/andybalholm/brotli v1.2.1
|
|
github.com/gin-contrib/cors v1.7.6
|
|
github.com/gin-gonic/gin v1.11.0
|
|
github.com/imroc/req/v3 v3.57.0
|
|
github.com/tidwall/gjson v1.19.0
|
|
)
|
|
|
|
require (
|
|
github.com/bytedance/gopkg v0.1.4 // indirect
|
|
github.com/bytedance/sonic v1.15.1 // indirect
|
|
github.com/bytedance/sonic/loader v0.5.1 // indirect
|
|
github.com/cloudwego/base64x v0.1.7 // indirect
|
|
github.com/gabriel-vasile/mimetype v1.4.13 // indirect
|
|
github.com/gin-contrib/sse v1.1.1 // indirect
|
|
github.com/go-playground/locales v0.14.1 // indirect
|
|
github.com/go-playground/universal-translator v0.18.1 // indirect
|
|
github.com/go-playground/validator/v10 v10.30.2 // indirect
|
|
github.com/goccy/go-json v0.10.6 // indirect
|
|
github.com/goccy/go-yaml v1.19.2 // indirect
|
|
github.com/google/go-querystring v1.2.0 // indirect
|
|
github.com/icholy/digest v1.1.0 // indirect
|
|
github.com/json-iterator/go v1.1.12 // indirect
|
|
github.com/klauspost/compress v1.18.6 // indirect
|
|
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
|
|
github.com/leodido/go-urn v1.4.0 // indirect
|
|
github.com/mattn/go-isatty v0.0.22 // indirect
|
|
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
|
|
github.com/modern-go/reflect2 v1.0.2 // indirect
|
|
github.com/pelletier/go-toml/v2 v2.3.1 // indirect
|
|
github.com/quic-go/qpack v0.6.0 // indirect
|
|
github.com/quic-go/quic-go v0.57.1 // indirect
|
|
github.com/refraction-networking/utls v1.8.2 // indirect
|
|
github.com/tidwall/match v1.2.0 // indirect
|
|
github.com/tidwall/pretty v1.2.1 // indirect
|
|
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
|
|
github.com/ugorji/go/codec v1.3.1 // indirect
|
|
golang.org/x/arch v0.27.0 // indirect
|
|
golang.org/x/crypto v0.52.0 // indirect
|
|
golang.org/x/net v0.55.0 // indirect
|
|
golang.org/x/sys v0.45.0 // indirect
|
|
golang.org/x/text v0.37.0 // indirect
|
|
google.golang.org/protobuf v1.36.11 // indirect
|
|
)
|