Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,9 @@ go mod download
go run main.go # reads ./config/config.yaml, then ./config.yaml, then /etc/e-sp-line2
```

The API listens on `:8080`. Before you expose it beyond localhost, change `jwt.secret` in the config — see [SECURITY.md](SECURITY.md).
The API listens on `:8080`. Before you expose it beyond localhost, change `jwt.secret` in the config — see [SECURITY.md](SECURITY.md). Startup warns while the placeholder is still in place.

The default config already carries `server` timeouts and a body cap; do not delete them. Serving the UI from another origin needs that origin in `security.allowed_origins` (it gates CORS *and* the WebSocket upgrade) — `ESP_SECURITY_ALLOWED_ORIGINS` overrides the list without editing the file.

```bash
make build # build to bin/e-sp-line2
Expand Down
8 changes: 5 additions & 3 deletions .github/SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,18 +52,20 @@ Out of scope:
### Backend

- Passwords are stored as bcrypt hashes, never in plaintext.
- API tokens are JWTs signed with HS256 and carry an expiry (`jwt.expiration`, 24 hours by default). Rotate `jwt.secret` in `backend/config/config.yaml` per deployment.
- API tokens are JWTs signed with HS256 and carry an expiry (`jwt.expiration`, 24 hours by default). Rotate `jwt.secret` in `config/config.yaml` per deployment; startup logs a WARN while the shipped placeholder is still in place.
- The `/api/v1` surface except `/auth/register` and `/auth/login` sits behind `AuthRequired`. `AdminRequired` exists but is not wired into any route yet.
- An adapter's `security` policy declares `sensitive_fields` (for Taobao and Xianyu, `cookie`) so a manifest can mark what must not be echoed back.
- CORS echoes a `security.allowed_origins` member instead of a wildcard, and the WebSocket upgraders (bridge, app and adapter gateway) check the same list. A cross-origin socket is refused with 403 before the upgrade, and both sockets require a valid JWT during the handshake (401 otherwise).
- The HTTP server runs with a header timeout, read/write/idle timeouts and a request-body cap, so a stalled or oversized request cannot pin a connection.

### What is not there yet

Be aware of these when you deploy, and please send a report if one of them bites you:

- The adapter instance `config` column stores its Cookie as plain JSON in the database, despite the `encrypted_fields` declaration. Treat the SQLite/PostgreSQL file as a secret.
- `AuthRequired` is still a placeholder: it checks the presence and `Bearer` shape of the header but does not yet verify the JWT signature, and it sets a fixed `user-123` identity. Do not expose the API to the internet and assume per-user isolation until that lands.
- CORS sends `Access-Control-Allow-Origin: *` with `Access-Control-Allow-Credentials: true`, and the WebSocket upgrader accepts any origin. Put the front end behind the same origin as the API in production.
- The default `jwt.secret` is a placeholder; change it before you deploy.
- The default `jwt.secret` is a placeholder; startup only warns, it does not refuse to boot. Change it before you deploy.
- `security.allowed_origins` defaults to `localhost` only. A real deployment must list its front-end origin (or set `ESP_SECURITY_ALLOWED_ORIGINS`); a `*` is not accepted, the list is matched exactly.

### Front end

Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -42,3 +42,6 @@ config/config.local.yaml
.vscode/
*.swp
.DS_Store

# Local scratch (demos, temp DBs, tool cache)
.tmp/
11 changes: 9 additions & 2 deletions adapters/taobao/esp_bridge.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
import asyncio
import json
import time
import urllib.parse
import uuid
from dataclasses import dataclass, field
from typing import Any, Callable, Dict, List, Optional
Expand Down Expand Up @@ -188,9 +189,15 @@ def __init__(
self.running = False

def _ws_url(self) -> str:
"""构造后端适配器 WebSocket 地址。"""
"""构造后端适配器 WebSocket 地址。

token 走 query 参数:后端在升级握手前校验它,缺失直接回 401。
"""
base = self.backend_url.replace("http://", "ws://").replace("https://", "wss://")
return f"{base}/ws/adapter?instance_id={self.instance_id}"
url = f"{base}/ws/adapter?instance_id={self.instance_id}"
if self.token:
url += f"&token={urllib.parse.quote(self.token, safe='')}"
return url

def _tag(self) -> str:
"""短标签:instance_id 前 8 位,便于在日志中区分多个桥。"""
Expand Down
10 changes: 9 additions & 1 deletion adapters/xianyu/esp_bridge.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
import asyncio
import json
import time
import urllib.parse
import uuid
from dataclasses import dataclass, field
from typing import Any, Dict, Optional, Callable
Expand Down Expand Up @@ -170,8 +171,15 @@ def __init__(
self.running = False

def _ws_url(self) -> str:
"""构造后端适配器 WebSocket 地址。

token 走 query 参数:后端在升级握手前校验它,缺失直接回 401。
"""
base = self.backend_url.replace("http://", "ws://").replace("https://", "wss://")
return f"{base}/ws/adapter?instance_id={self.instance_id}"
url = f"{base}/ws/adapter?instance_id={self.instance_id}"
if self.token:
url += f"&token={urllib.parse.quote(self.token, safe='')}"
return url

def _tag(self) -> str:
"""短标签:instance_id 前 8 位,便于在日志中区分多个桥。"""
Expand Down
22 changes: 22 additions & 0 deletions config/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,31 @@ log_level: "info" # debug, info, warn, error

# JWT settings
jwt:
# The shipped value is a placeholder: while it is in place startup logs a
# WARN and anyone can forge a token. Generate one per deployment, e.g.
# openssl rand -base64 32
secret: "change-this-secret-in-production"
expiration: 24 # hours

# HTTP server settings — all seconds, 0 disables a timeout.
server:
read_header_timeout: 10
read_timeout: 60
write_timeout: 120
idle_timeout: 120
max_body_bytes: 4194304 # 4 MiB

# Browser security. One allowlist, used by CORS *and* the WebSocket Origin
# check. Add the real front-end Origin when the UI is served from elsewhere;
# ESP_SECURITY_ALLOWED_ORIGINS (comma-separated) overrides this list. An empty
# list means "allow any Origin" (development fallback).
security:
allowed_origins:
- "http://localhost:3000"
- "http://127.0.0.1:3000"
- "http://localhost:8080"
- "http://127.0.0.1:8080"

# Adapter settings
adapter:
heartbeat_interval: 30 # seconds
Expand Down
121 changes: 118 additions & 3 deletions docs/developer-guide/api-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,8 @@ Public. Returns the message field contract that [message-fields.md](./message-fi

**DELETE** `/platforms/:id`

A missing id answers **404**.

**Response**:
```json
{
Expand Down Expand Up @@ -338,6 +340,8 @@ Public. Returns the message field contract that [message-fields.md](./message-fi

**DELETE** `/adapters/:id`

A missing id answers **404**.

**Response**:
```json
{
Expand Down Expand Up @@ -391,6 +395,9 @@ Public. Returns the message field contract that [message-fields.md](./message-fi
"name": "Store A",
"status": "running",
"config": "{}",
"last_heartbeat": "2024-01-01T00:05:00Z",
"message_count": 128,
"error_count": 3,
"created_at": "2024-01-01T00:00:00Z"
}
],
Expand Down Expand Up @@ -439,6 +446,9 @@ Public. Returns the message field contract that [message-fields.md](./message-fi
"name": "Store A",
"status": "running",
"config": "{}",
"last_heartbeat": "2024-01-01T00:05:00Z",
"message_count": 128,
"error_count": 3,
"session": {
"connected_at": "2024-01-01T00:00:00Z",
"last_heartbeat": "2024-01-01T00:05:00Z"
Expand All @@ -448,6 +458,11 @@ Public. Returns the message field contract that [message-fields.md](./message-fi
}
```

`last_heartbeat` is `null` until the adapter starts or its bridge connects.
`message_count` / `error_count` are instance-lifetime counters, incremented on
inbound message persistence and on failed persistence respectively; the
Monitoring page reads exactly these three fields.

### Update instance

**PUT** `/instances/:id`
Expand Down Expand Up @@ -476,6 +491,8 @@ Public. Returns the message field contract that [message-fields.md](./message-fi

**DELETE** `/instances/:id`

A missing id answers **404**.

**Response**:
```json
{
Expand All @@ -485,6 +502,42 @@ Public. Returns the message field contract that [message-fields.md](./message-fi

## Message endpoints

### Create (ingest) a message

**POST** `/messages`

The entry point for a message arriving from outside the bridge socket. The event
is persisted and then routed: if a rule matches and a target is connected the
frame is delivered, and the stored `status` records how far it got
(`received` → no rule matched, `routed` → matched but nobody connected,
`delivered` → at least one connection accepted it).

**Request body**:
```json
{
"platform_id": "xianyu",
"instance_id": "instance-001",
"conversation_id": "conv-001",
"sender_id": "user-123",
"sender_name": "Buyer A",
"message_type": "text",
"message_content": "Hello",
"idempotency_key": "xianyu-msg-001"
}
```

`idempotency_key` is optional. When it is omitted a key is generated, so repeated
keyless messages never collide.

**Responses**:

| Code | Meaning |
| --- | --- |
| `201` | Created and (attempted) routed; body is the event |
| `200` | Same `idempotency_key` already stored; body is the **original** event |
| `400` | `platform_id` / `instance_id` / `message_type` missing |
| `401` | No valid token |

### List messages

**GET** `/messages`
Expand Down Expand Up @@ -545,6 +598,8 @@ Public. Returns the message field contract that [message-fields.md](./message-fi

**POST** `/messages/:id/ack`

A missing id answers **404**.

**Response**:
```json
{
Expand Down Expand Up @@ -743,6 +798,8 @@ Public. Returns the message field contract that [message-fields.md](./message-fi

**DELETE** `/routes/:id`

A missing id answers **404**.

**Response**:
```json
{
Expand All @@ -752,9 +809,26 @@ Public. Returns the message field contract that [message-fields.md](./message-fi

## WebSocket interface

Both sockets authenticate during the HTTP handshake, before the upgrade. A
missing or invalid token is answered with **401** as a plain HTTP response, and
an `Origin` outside the allowlist with **403**, so a failure is readable rather
than an immediately-closed socket.

The token is accepted in any of three forms, because the callers differ:

| Transport | Caller |
|---|---|
| `?token=<jwt>` query parameter | the Python bridges (they cannot set headers on connect) |
| `Authorization: Bearer <jwt>` header | server-to-server clients, `websocat`, tests |
| `Sec-WebSocket-Protocol: espl, <jwt>` | browsers — the only header a page can set on `WebSocket` |

`Origin` is checked against `security.allowed_origins` (the same list the CORS
middleware uses; `ESP_SECURITY_ALLOWED_ORIGINS` overrides it). A request without
an `Origin` header is a non-browser client and is allowed through.

### Adapter WebSocket

**URL**: `ws://localhost:8080/ws/adapter?instance_id=<instance_id>`
**URL**: `ws://localhost:8080/ws/adapter?instance_id=<instance_id>&token=<jwt>`

**Message format**:
```json
Expand All @@ -776,7 +850,7 @@ Public. Returns the message field contract that [message-fields.md](./message-fi

### App WebSocket

**URL**: `ws://localhost:8080/ws/app?app_id=<app_id>`
**URL**: `ws://localhost:8080/ws/app?app_id=<app_id>&token=<jwt>`

**Message format**:
```json
Expand All @@ -796,17 +870,58 @@ Public. Returns the message field contract that [message-fields.md](./message-fi
}
```

## CORS

Browsers get the request `Origin` echoed back in
`Access-Control-Allow-Origin` when it appears in `security.allowed_origins`
(defaults cover `localhost:3000` and `localhost:8080`); `Vary: Origin` is set so
caches key on it. Because the value is a concrete origin rather than `*`,
`Access-Control-Allow-Credentials: true` is meaningful and a future Cookie-based
session keeps working. Requests from an unlisted origin still reach the handler
but carry no CORS headers, so the browser blocks the response. Requests without
an `Origin` (curl, the Python bridges) are unaffected.

## Limits

- Request bodies are capped by `server.max_body_bytes` (default 4 MiB); an
oversized body is refused with **413**.
- The HTTP server enforces `server.read_header_timeout` (default 10s),
`read_timeout`, `write_timeout` and `idle_timeout`. All are seconds; `0`
disables the corresponding timeout.

## Error codes

Every error body is `{"error": "<reason>"}`.

| Error code | Description |
|--------|------|
| 400 | Invalid request parameters |
| 400 | Invalid request parameters (binding failure, failed validation) |
| 401 | Unauthenticated |
| 403 | Forbidden |
| 404 | Resource not found |
| 409 | Resource conflict |
| 413 | Request body too large |
| 500 | Internal server error |

Mapping rules, so a client can branch on the status instead of parsing text:

- **404** — the URL names an id that does not exist. Applies to every `GET`,
`PUT`/`PATCH`, `DELETE` and to `POST /messages/:id/ack`: operating on a
missing resource is not a success. The body is the generic
`resource not found`; the id is in the server log.
- **409** — the write collides with existing state and retrying unchanged will
not help: duplicate username (`POST /auth/register`, `POST /users`),
duplicate platform code, demoting/disabling/deleting the last admin. The
body carries the specific reason, e.g. `username already exists`.
- **400** — the request itself is wrong: malformed JSON, missing required
field, password too short, unknown role/status, starting an instance with no
cookie or while it is still initializing.
- **500** — storage or unexpected failure. The body is the raw error.

`DELETE` on a missing id used to answer `200` with `"… deleted"`; it now
answers `404`. Clients that treated the old 200 as "done" keep working, ones
that re-fetched afterwards get a truthful answer.

## Rate limiting

- Default limit: 100 requests/minute
Expand Down
8 changes: 8 additions & 0 deletions docs/developer-guide/bridge-protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,14 @@ ws://<esp-host>:<esp-port>/ws/adapter?instance_id=<INSTANCE_ID>
| `send_image` | 发送图片消息 | `cid`、`toid`、`image_url` 等 |

> 桥进程收到指令后调用电商平台 API 发送,发送结果通过日志记录。
>
> **字段形状是扁平的一级 `command_type`/`type` 加 `payload.cid`/`payload.toid`/`payload.text`**,
> 不是 ESPL v3 信封。桥的解析见 `adapters/xianyu/main.py:197-220` 与
> `adapters/taobao/main.py:228-250`:它在顶层取 `command_type`,再取 `payload`,
> 缺 `cid`/`toid` 直接告警丢弃。后端由 `hub.SendCommand` 按本形状组帧。
>
> 下发失败(该实例没有连接)时命令进入 `retrying` 并由 30 秒的扫描重试,
> 不会静默停在 `created`。

## 4. 心跳与重连

Expand Down
Loading
Loading