aars/README.md

# AARS - Attendance & Accident Record System

出勤及意外記錄系統

## 🚀 快速開始

### 方式一：Docker 部署（推薦）

```bash
# 复制到 VPS
scp -r ~/aars root@187.127.116.15:/root/

# SSH 到 VPS
ssh root@187.127.116.15

# 進入目录并运行
cd /root/aars
chmod +x setup.sh
./setup.sh
```

访问：`http://187.127.116.15:18775`

### 方式二：本地开发

```bash
# Backend
cd backend
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
python test_backend.py  # 测试
uvicorn main:app --reload --port 8000

# Frontend (另一个terminal)
cd frontend
npm install
npm run dev
```

## 🔐 默认账户

- **Email:** admin@aars.hk
- **Password:** admin123

⚠️ 首次登录后请立即修改密码！

## 📋 功能

### 出勤 Attendance
- ✅ 列表视图 (Filter/Sort/分页)
- ✅ 新增/编辑/删除记录
- ✅ Excel 批量导入（自动去重）
- ✅ 导出 Excel / PDF
- ✅ 自定义字段

### 意外 Accident
- ✅ 列表视图 (Filter/Sort/分页)
- ✅ 新增/编辑/删除记录
- ✅ Excel 批量导入（自动去重）
- ✅ 导出 Excel / PDF
- ✅ 自定义字段

### Dashboard
- 📊 统计卡片（总计/今日/本月）
- 📈 趋势图表
- ⚠️ 最近意外列表

## 🏗️ 技术栈

| Layer | Technology |
|-------|------------|
| Frontend | React 18 + Vite + TailwindCSS |
| Backend | FastAPI + SQLAlchemy |
| Database | SQLite |
| Charts | Recharts |
| Excel | openpyxl + xlsxwriter |
| PDF | ReportLab |

## 📁 项目结构

```
aars/
├── backend/
│   ├── main.py          # FastAPI app
│   ├── models.py        # Database models
│   ├── schemas.py       # Pydantic schemas
│   ├── auth.py          # JWT auth
│   ├── database.py      # SQLite setup
│   └── requirements.txt
├── frontend/
│   ├── src/
│   │   ├── pages/       # React pages
│   │   ├── components/  # Reusable components
│   │   └── context/     # Auth context
│   └── ...
├── docker-compose.yml
├── Dockerfile
└── SPEC.md
```

## 🔧 配置

### 端口
- 默认: `18775`
- 修改: 编辑 `docker-compose.yml`

### JWT Secret
- 开发: 自动生成
- 生产: 修改 `backend/secret.py`

## 📝 API 文档

启动后访问：`http://localhost:8000/docs` (Swagger UI)

## 🐛 常见问题

**Q: Docker build 失败？**
```bash
docker compose down
docker compose build --no-cache
docker compose up -d
```

**Q: 无法登录？**
检查浏览器 console 的 CORS 错误，或确认 JWT secret 未变更。

**Q: Excel 导入失败？**
确保 Excel 文件第一行是表头，且日期格式为 `YYYY-MM-DD`。

## 📊 Attendance System

### Data Source
- **Primary**: SQLite table `attendance_records` (SQLAlchemy ORM)
- **Legacy**: Excel file at `/app/data/attendance.xlsx` (still saved on upload for export compatibility)

### DB Schema (`attendance_records`)

| Column | Type | Notes |
|--------|------|-------|
| `id` | INTEGER PK | auto |
| `employee_name` | VARCHAR(255) | index |
| `company`, `department` | VARCHAR(100) | nullable |
| `date` | DATE | index |
| `weekday` | VARCHAR(20) | e.g. "Friday" |
| `check_in`, `check_out` | DATETIME | nullable |
| `shift_code` | VARCHAR(20) | e.g. "S7a" |
| `status_code` | VARCHAR(20) | `normal` / `late` / `early` / `late_early` / `late_ot` / `early_ot` / `ot` / `missing` / `abnormal` |
| `status_text` | VARCHAR(100) | Chinese e.g. 「遲到46分 / OT60分」 |
| `late_minutes`, `early_minutes`, `ot_minutes` | INTEGER | computed |
| `expected_in`, `expected_out`, `actual_in`, `actual_out` | VARCHAR(10) | "HH:MM" string |
| `is_manually_edited` | BOOLEAN | **preserves manual edits from import overwrite** |
| `raw_data` | JSON | source row |
| `shift_id` | INTEGER FK | → `shifts.id` |

UNIQUE constraint on `(employee_name, date)` — one record per staff per day.

### Shifts Table (`shifts`)

- 11 rows pre-loaded (S7a, S8a, ...)
- Columns: `shift_code` + per-day (`mon_start`/`mon_end` ... `sun_start`/`sun_end`), all `HH:MM` strings
- Method: `Shift.get_schedule(weekday)` → `(start_str, end_str)` or `(None, None)` on day off

### Status Calculation (`calculate_attendance_status` in `main.py`)

| Status | Rule |
|--------|------|
| `missing` | No check-in or no check-out |
| `abnormal` | check_in == check_out (exact same second) |
| `late` | check_in > expected start (late_minutes = round(diff)) |
| `early` | check_out < expected end (early_minutes = round(diff)) |
| `ot` | check_out > expected end (ot_minutes = round(diff)) |
| `late_early`, `late_ot`, `early_ot` | combined codes |

Notes:
- `int()` was buggy → now uses `round()` (off-by-one fix)
- `18:00:00` exact default check (NOT `18:00:03` which is real swipe)

### API Endpoints (current, attendance scope)

| Method | Path | Source | Notes |
|--------|------|--------|-------|
| GET | `/api/dashboard/summary` | **SQL** | Returns `total_records`, `*_count`, `staff_count`, `by_status`, `by_department` (frontend-compatible fields) |
| GET | `/api/attendance/stats` | **SQL (alias)** | Per-employee: count + minutes + avg + max + consecutive_missing + last_attendance_date + attendance_rate |
| GET | `/api/attendance/records` | **SQL (alias)** | Supports `staff`, `date_from`, `date_to`, `status`, `search`, `sort_by`, `sort_order`, `page`, `per_page` |
| GET | `/api/attendance/{id}` | SQL | Single record by ID |
| PUT | `/api/attendance/{id}` | SQL | Update + mark `is_manually_edited=true` (protected from import) |
| POST | `/api/upload/attendance` | Excel + SQL | Save file + import to DB (skips records where `is_manually_edited=true`) |
| GET | `/api/export/attendance/excel` | **SQL → xlsxwriter** | On-the-fly 16-col xlsx with status colors |

### Excel Export Columns (16)

| Col | Name | Notes |
|-----|------|-------|
| A | Staff Name | |
| B | Company | |
| C | Department | |
| D | Date | ISO `YYYY-MM-DD` |
| E | Weekday | |
| F | Shift | |
| G | Expected In | |
| H | Expected Out | |
| I | Actual In | |
| J | Actual Out | |
| **K** | **Status Code** | **color-coded per status** |
| L | Status Text | Chinese |
| M | Late Min | |
| N | Early Min | |
| O | OT Min | |
| P | Manual Edit | "Yes" if manually edited |

### Status Colors (xlsxwriter)

| Status | BG | Font |
|--------|------|------|
| Header | `#DBEAFE` (light blue) | `#1E3A8A` (dark blue, bold) — frozen top row |
| normal | `#D1FAE5` (green) | `#065F46` |
| late / late_early | `#FEF3C7` (amber) | `#92400E` |
| late_ot | `#FED7AA` (orange) | `#7C2D12` |
| early | `#DBEAFE` (blue) | `#1E40AF` |
| early_ot | `#E0E7FF` (indigo) | `#1E40AF` |
| ot | `#EDE9FE` (purple) | `#5B21B6` |
| abnormal | `#FEE2E2` (red) | `#7F1D1D` |
| missing | `#E5E7EB` (gray) | `#374151` |

### Frontend Components

- **Dashboard** (`pages/Dashboard.jsx`):
  - 8 stats cards (total / normal / late / early / OT / missing / abnormal / staff)
  - 4 leaderboards (Late / OT / Missing / Attendance Rate)
  - 5 date presets (今日 / 本週 / 上週 / 本月 / 清除)
  - Per-staff detail row: avg / max / consecutive_missing / last_attendance_date / attendance_rate
- **Attendance List** (`pages/attendance/List.jsx`):
  - Table with filters (status, date range, staff, search)
  - Pagination + sort
  - "Dashboard" back-link button (`to="/"`, NOT `to="/dashboard"`)
- **Attendance Detail** (`pages/attendance/Detail.jsx`):
  - Per-record view + edit

### Manual Edit Flow

1. User edits record via UI (Detail page) → `PUT /api/attendance/{id}` → marks `is_manually_edited=true`
2. Next upload of Excel → import loop checks `is_manually_edited` → skips this record (preserves manual change)

### Known Data Quirks

- `expected_in` / `expected_out` stored as strings ("11:00") for direct UI display
- `actual_in` / `actual_out` same — strings, NOT datetimes
- `Shift.get_schedule()` returns strings — caller must `datetime.strptime(start_str, "%H:%M").time()` before passing to `calculate_attendance_status`

### Files

- Backend: `/root/aars/backend/main.py` (~2100 lines), `models.py`, `database.py`, `schemas.py`, `auth.py`
- DB file: `/root/aars/data/aars.db` (SQLite, bind-mounted into container)
- Frontend: `/root/aars/frontend/src/pages/Dashboard.jsx`, `pages/attendance/List.jsx`, `pages/attendance/Detail.jsx`
- Excel data: `/root/aars/data/attendance.xlsx` (legacy), `roster.xlsx`, `accident.xlsx`

### Computed Backend Data Caveats

- **Status counts are dynamic** — computed from `attendance_records` table at query time
- **No materialized views** — query heavy if data grows beyond ~100k records
- **Roster changes affect historical data** — if you update `Shift.mon_start` for S7a, all past records with `shift_code="S7a"` are NOT auto-recalculated (run recompute manually if needed)

## 🐛 Debug Logging

Structured JSON logs written to `/root/aars/logs/` (bind-mounted into container at `/app/logs/`). Designed for production debugging — NOT compliance/audit.

### Files

| File | Contents | Retention |
|------|----------|-----------|
| `app-YYYY-MM-DD.jsonl` | General INFO+ events | 7 days |
| `error-YYYY-MM-DD.jsonl` | Server + client errors (with stack) | 30 days |
| `access-YYYY-MM-DD.jsonl` | One entry per HTTP request | 7 days |

Files auto-rotate daily via in-process `DailyJSONLHandler`. Old files deleted by `/root/aars/scripts/logs-cleanup.sh` (cron at 03:00 recommended).

### Log Schema

Every entry is single-line JSON:

```json
{
  "ts": "2026-07-17T08:30:00.123Z",
  "level": "info",
  "service": "aars-backend",
  "env": "production",
  "request_id": "abc123-...",
  "route": "/api/dashboard/summary",
  "method": "GET",
  "status": 200,
  "duration_ms": 114,
  "msg": "human readable summary",
  "error": { "name": "...", "message": "...", "stack": "..." }
}
```

- `ts`: ISO 8601 UTC
- `level`: `debug` / `info` / `warn` / `error`
- `service`: `aars-backend` (frontend errors tagged with `"source": "client"` in `context`)
- `env`: hardcoded `production`
- `request_id`: from `X-Request-ID` header (auto-generated if missing, validated if present)
- Sensitive fields auto-redacted (see below)

### Sensitive Redaction

Backend redactor scrubs these substrings from any log payload (dicts/strings walked recursively):

- `password`, `passwd`, `pwd`, `pass`
- `token`, `access_token`, `refresh_token`, `id_token`, `jwt`, `bearer`
- `api_key`, `apikey`, `secret`, `client_secret`
- `authorization`, `cookie`, `set-cookie`, `session`
- `database_url`, `db_url`, `connection_string`

JWT regex: `eyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+` → `[REDACTED:JWT]`

**Verification**:

```bash
grep -l password /root/aars/logs/*.jsonl   # should return nothing
grep admin123 /root/aars/logs/*.jsonl     # should return nothing
```

### Reproducing an Error (workflow for ITdog)

1. User reports an issue with `request_id` (visible in browser DevTools → Network → Response Headers → `X-Request-ID`)
2. SSH to VPS:
   ```bash
   ssh root@187.127.116.15
   ```
3. Search by request_id across all 3 files:
   ```bash
   grep '"request_id":"paste-id-here"' /root/aars/logs/{app,error,access}-2026-07-17.jsonl | jq .
   ```
4. Search by route / status / message:
   ```bash
   grep '"route":"/api/dashboard/summary"' /root/aars/logs/error-*.jsonl | jq .
   grep '"status":500' /root/aars/logs/access-*.jsonl | jq .
   grep 'login failed' /root/aars/logs/app-*.jsonl | jq .
   ```
5. Live tail for new errors:
   ```bash
   tail -f /root/aars/logs/error-*.jsonl | jq .
   ```

### Live Tailing

```bash
tail -f /root/aars/logs/app-2026-07-17.jsonl | jq .     # app events
tail -f /root/aars/logs/access-2026-07-17.jsonl | jq .  # HTTP requests
tail -f /root/aars/logs/error-2026-07-17.jsonl | jq .   # errors
```

(Requires `jq` — `apt install -y jq` on VPS if not present.)

### npm Scripts (from frontend dir)

```bash
cd /root/aars/frontend
npm run logs:tail       # tail app log
npm run logs:errors     # tail error log
npm run logs:access     # tail access log
npm run logs:search -- '"route":"/api/dashboard"'    # search all 3 files
npm run logs:clean      # manually trigger retention cleanup
```

### Triggering a Test Client Error

From browser console:
```js
import('/src/lib/logger.js').then(m => m.logger.error('manual test', { stack: 'fake stack' }))
```

Or via curl (no auth required):
```bash
curl -X POST https://excel.donton.cloud/api/client-logs \
  -H "Content-Type: application/json" \
  -H "X-Request-ID: my-debug-test" \
  -d '{"level":"error","msg":"manual curl test","stack":"foo","page_url":"https://excel.donton.cloud/dashboard","app_version":"1.0.0"}'
```

Then verify:
```bash
grep "my-debug-test" /root/aars/logs/error-*.jsonl | jq .
```

### Manual Retention Cleanup

```bash
npm run logs:clean
# or directly
bash /root/aars/scripts/logs-cleanup.sh
```

To schedule daily, add to crontab on the host:
```cron
0 3 * * * /root/aars/scripts/logs-cleanup.sh >> /var/log/aars-cleanup.log 2>&1
```

### Architecture

**Backend** (FastAPI stdlib only, no new deps):
- `backend/logging_config.py`:
  - `JSONFormatter` — formats `LogRecord` → single-line JSON
  - `DailyJSONLHandler` — in-process daily file rotation (not stdlib `TimedRotatingFileHandler` which has awkward filename semantics)
  - `_scrub(obj)` — recursive dict walker + JWT regex
  - `ContextVar` `request_id_var` + `user_id_var` for cross-function context
  - Console handler with `ContextFilter` for human-readable stdout output
- `backend/middleware.py`:
  - `RequestIDMiddleware` — generate uuid4 if `X-Request-ID` missing, validate if present (`[A-Za-z0-9_-]{1,128}` regex)
  - `AccessLogMiddleware` — log every request with route/method/status/duration_ms
  - `unhandled_exception_handler` — catch-all 500 with stack trace

**Frontend** (vanilla JS, no extra deps):
- `frontend/src/lib/logger.js`:
  - Buffer (50 entries max), flush every 10s + on `pagehide` + on visibility hidden
  - `navigator.sendBeacon` (preferred) → `fetch` fallback
  - Captures `console.error`, `unhandledrejection`, ErrorBoundary errors
  - Redacts same keys as backend (client-side filter)
- `frontend/src/api.js`:
  - Axios interceptor — generate `X-Request-ID` per request, capture from response
- `frontend/src/components/ErrorBoundary.jsx`:
  - React error boundary that posts to `/api/client-logs`
- `frontend/src/main.jsx`:
  - Wraps app in ErrorBoundary, installs global error handlers

**Server endpoint**:
- `POST /api/client-logs` — no auth, validates via `ClientLogEntry` Pydantic schema, writes to error log with `source: client` tag

### Excluded (by design)

- ❌ Rate limiting on `/api/client-logs` — debug use, no DDoS concern
- ❌ PII pattern scrubbing (emails, HKID) — DB has no such data
- ❌ Lint / type / unit tests — user said no testing around
- ❌ Compliance / audit logging — no requirements
- ❌ Centralized log aggregation (ELK, Datadog) — VPS-only, SSH access suffices

### Files

| Path | Purpose |
|------|---------|
| `/root/aars/backend/logging_config.py` | Formatter + handler + redactor |
| `/root/aars/backend/middleware.py` | RequestID + AccessLog + exception handler |
| `/root/aars/scripts/logs-cleanup.sh` | Retention cron (run daily) |
| `/root/aars/logs/app-*.jsonl` | App events (7d) |
| `/root/aars/logs/error-*.jsonl` | Errors + stack traces (30d) |
| `/root/aars/logs/access-*.jsonl` | HTTP access log (7d) |
| `/root/aars/frontend/src/lib/logger.js` | Client logger |
| `/root/aars/frontend/src/api.js` | Axios + X-Request-ID interceptor |
| `/root/aars/frontend/src/components/ErrorBoundary.jsx` | React error boundary |

### Known Issues

1. **`jq` not pre-installed on VPS** — `apt install -y jq` required for pretty-printed logs
2. **Crontab for `logs-cleanup.sh` not auto-installed** — manually add `0 3 * * *` cron entry on host
3. **Container healthcheck always reports unhealthy** — uses `/api/auth/me` (returns 403 without auth). Cosmetic only, doesn't affect functionality.