# Maintenance Memory — Shopify Balance Adjustments, Disputes & Token Auth (Pascucci)

**Created:** 2026-06-01
**Scope tenant:** Pascucci USA Inc (PUI) — `tenant_id=837eebaa-1ae8-5b79-95c3-26649fb25c42`, shopify `integration_instance_id=3c7e91d9-8a2d-5382-9da0-dd2d7bfb8213`, shop `gmz3q2-41.myshopify.com`.
**Design doc:** `plans/SHOPIFY_BALANCE_ADJUSTMENTS_AND_DISPUTES_V1.md`
**Status:** **DEPLOYED + verified 2026-06-01.** Rebuilt/restarted core-api, connector-shopify, connector-erpnext,
jobs, jobs-beat. Migration `0036_shopify_disputes` applied. The 2 ERPNext accounts exist
(`Chargeback Losses - PUI` Expense, `Shopify Disputed Funds - PUI` Asset). client_credentials token auth live
(disputes endpoint 200). All 4 payouts reconcile cent-exact through the deployed core-api. Dispute poller
deployed (beat 02:50 daily) and tracked dispute #1083 (`under_review`). REMAINING OPERATIONAL STEP: categorize
(export) the 4 payouts in the UI — each matched to its bank deposit (`canonical_transaction_id`); previews are
verified, the export posts the JE. The dispute LOST-reclass auto-post will fire when a dispute actually resolves
(none lost yet; #1083 is open) — its JE shape is dry-run-verified but has NOT posted to the ledger.

This file documents TWO tasks done 2026-06-01 for future maintenance. Read it before touching Shopify
payout routing or the connector's auth.

---

## TASK 1 — Shopify balance-adjustment routing (the "credit/debit/dispute won't ingest" bug)

### Root cause (verified)
`PASCUCCI_SHOPIFY_TXN_TYPE_MAP` was keyed on Shopify's **GraphQL fine-grained** txn enum
(`dispute_withdrawal`, `tax_adjustment_credit`, …). But the connector reads the **REST** endpoint
`/shopify_payments/balance/transactions.json`, whose coarse `type` is `charge|refund|payout|credit|debit|dispute`.
`credit`/`debit`/`dispute` had no map entry → `_build_shopify_preview_payload` refused them → HTTP 412 →
payout could not be categorized. The real discriminator for `credit`/`debit` is the **`adjustment_reason`**
field (observed: `shop_cash`, `tax_adjustment`), which the connector was dropping.

### Fix = canonical routing keys
- **`shopify_routing_key(txn)`** (`services/product_classifier.py`): `credit`/`debit` with
  `source_type=adjustment` → `adjustment:<adjustment_reason>`; everything else → bare lower `type`.
- New `TxnRouting.kind` discriminator + map entries:
  - `adjustment:shop_cash` → kind `shop_cash`, `requires_order_context=True`
  - `adjustment:tax_adjustment` → kind `tax_adjustment`, `requires_order_context=True`
  - `dispute` → kind `dispute`
- New account constants: `ACCT_SHOPIFY_DISPUTED_FUNDS="Shopify Disputed Funds - PUI"`,
  `ACCT_CHARGEBACK_LOSSES="Chargeback Losses - PUI"`.

### Per-reason GL treatment (all verified against live orders)
- **shop_cash** (e.g. credit +20.00): Shop Cash is Shopify-funded cashback the customer applied at
  checkout; the card `charge` is NET of it and Shopify reimburses the merchant. So it is **revenue
  completion**, booked to the order's family **Product Sales** (same account as the charge), NOT a
  discount. Its processing fee flows to CC Processing Fees. Verified on #1132: charge 86.80 +
  shop_cash 20.00 = 106.80 order total.
- **tax_adjustment** (e.g. debit −7.80): Shopify sales-tax true-up. Routed to the Sales Tax leg chosen
  by the order's `channel_liable` split via `_split_tax_adjustment` (`routers/recon.py`). For Pascucci
  (CA marketplace facilitator) all observed are channel-liable → **Sales Tax Already Paid - PUI**, which
  exactly nets the charge's tax credit. Fail-closed: if the order breakdown is unavailable, the txn is
  REFUSED (412) rather than guessing the leg.
- **dispute** (e.g. −99.00 principal, 15.00 fee): principal → **Shopify Disputed Funds - PUI** (transition
  asset), fee → **Chargeback - PUI** via `special_fee_mapping` (kept out of the generic CC-fees bucket but
  still counted in `total_fees` so the deposit reconciles to the cent). See Task 1b for resolution.

### Key files changed (Task 1)
- `finanly/services/core-api/src/finanly_core_api/services/product_classifier.py`
  - new accounts; `TxnRouting.kind`; map entries; `shopify_routing_key()`.
- `finanly/services/core-api/src/finanly_core_api/routers/recon.py`
  - import new symbols; `_split_tax_adjustment()`; dispatch on `routing_key`+`kind`
    (dispute / shop_cash / tax_adjustment branches); `special_fee_mapping` (dispute fee → Chargeback,
    still in `total_fees`); cache version bumped to `2026-06-01.6`.
  - `/unmapped-categories` endpoint now derives canonical keys from `transactions`.
- `finanly/services/core-api/src/finanly_core_api/routers/internal_recon_digest.py`
  - daily email now reports canonical keys (`adjustment:shop_cash`, not `credit`) via
    `_shopify_unmapped_keys_for_item()`.

### Cent-exact invariant — UNCHANGED and preserved
`sum(gl_entries) == total_amount − total_fees == payout deposit`. Each split (charge/refund/tax_adjustment)
sums to its txn `amount` or returns None→refuse. Dispute fee stays in `total_fees` (deposit really drops
by it). Unknown `adjustment:<reason>` → refuse (412) + surfaced in the daily digest. Default-refuse intact.

### Dry-run evidence (2026-06-01, real `_build_shopify_preview_payload`, no writes)
4 blocked payouts all reconcile to the cent:
`131619618986`→94.13, `131711533226`→964.93, `131777036458`→1056.23, `131808755882`→536.93.
Harness: `temp/dryrun_shopify_adjustments.py` (+ `temp/dryrun_dispute_and_shopcash.py`).

---

## TASK 1b — Dispute (chargeback) lifecycle

New module `finanly/services/core-api/src/finanly_core_api/services/dispute_lifecycle.py`:
- `normalize_dispute_resolution(status)`: `won` → "won"; `lost`/`accepted` → "lost"; else None (open).
- `build_dispute_resolution_je(principal, resolution)`:
  - **won** → `None`. The returned principal arrives as a balance-txn on a FUTURE payout, which the
    same routing books `Cr Shopify Disputed Funds`, netting the parked debit to 0. Revenue stands; the
    non-refundable fee stays expensed. (ASSUMPTION to confirm on first real Won: that the return is a
    positive `dispute`-type balance-txn on a payout. No Won dispute existed in data on 2026-06-01.)
  - **lost** → standalone reclass JE (no bank anchor): `Dr Chargeback Losses - PUI 99 / Cr Shopify
    Disputed Funds - PUI 99`. Balanced. Dry-run verified for #1083 ($99).

### Lifecycle — REMAINING wiring (not yet built; specified for the next session)
1. **Model + migration** `0036_shopify_disputes` (head is `0035_recon_preview_cache`): table
   `shopify_disputes(dispute_id, order_id, payout_id, principal, fee, currency, status, resolution,
   opened_je_ref, resolved_je_ref, opened_at, resolved_at)` + RLS policy (mirror other tenant-scoped tables).
2. **Connector-erpnext posting kind** `reclass_journal_entry`: a standalone 2-line JE (voucher_type
   "Journal Entry", no Bank Transaction anchor). `journal_entry_split` does NOT fit — it requires a bank
   txn. Mirror the fx_revaluation JE creation path. Takes `lines:[{account,cost_center,debit,credit}]`.
3. **Resolve endpoint** `POST /v1/recon/internal/disputes/{id}/resolve` (idempotent key
   `dispute_id:resolution`, audited `recon.dispute_won|lost`): on `lost`, create an `ExportItem` with
   `posting_kind="reclass_journal_entry"` and the lines from `build_dispute_resolution_je`; on `won`,
   record only.
4. **Poller** Celery task in `finanly/services/jobs/.../tasks.py` + beat entry (`celery_app.py`,
   nightly ~02:xx): call connector `GET /v1/shopify/disputes`, diff status vs `shopify_disputes`,
   call the resolve endpoint on transitions. Requires the disputes scope (Task 2).

---

## TASK 2 — Robust Shopify token auth (client_credentials minting)

### Root cause
Finanly stored a STATIC `shopify.access_token` (`shpat_eeffbd…`, created 2026-02-16) whose granted
scopes froze at issue time = 41 scopes WITHOUT `read_shopify_payments_disputes`. Shopify's
`/admin/oauth/access_scopes.json` confirmed it; the disputes endpoint 403'd:
`"[API] This action requires merchant approval for read_shopify_payments_disputes scope."`
The PUI app's CONFIG has the scope (177 total), but a static token never picks up later scope additions.

### Fix = connector mints `client_credentials` tokens (always current scopes)
The PUI app supports the `client_credentials` grant at `POST /admin/oauth/access_token`
(see `/home/shopify/services/shopify-api.sh`). Tokens are short-lived (~24h, `expires_in=86399`) but
always reflect the app's CURRENT configured scopes.

**Connector change** (`finanly/services/connectors/shopify/src/finanly_connector_shopify/app.py`):
- Refactored `_get_secret` into `_fetch_secret_raw(..., optional=False)` + a thin `_get_secret`
  CHOKEPOINT: any request for key `shopify.access_token` routes through `_resolve_access_token`.
- `_resolve_access_token`: if `shopify.client_id` + `shopify.client_secret` secrets exist → mint+cache
  a client_credentials token; else fall back to the static `shopify.access_token` (other tenants).
- `_mint_client_credentials_token` + `_get_cached_or_mint_token`: in-process cache
  (`_TOKEN_CACHE` keyed by integration_instance_id), refresh `_TOKEN_REFRESH_BUFFER_S=300`s before
  expiry, `threading.Lock` so concurrent callers mint at most once, `force_refresh` for a 401 backstop.

### Why this is robust for the nightly sync
Minting is **lazy + inline**: every Shopify call resolves the token right before use, so there is no
separate refresh schedule that can drift. The 2am sync's first call mints fresh if the cache is stale or
absent (e.g. after a restart). Shopify returns the SAME valid token when re-minted inside its window →
no invalidation thrash across mints/replicas. Verified end-to-end: mint → cache hit → disputes scope
present (177) → disputes endpoint 200 → force_refresh still yields a working token.

### Secrets state (already stored 2026-06-01)
`integration_secrets` for the Pascucci shopify instance now has:
`shopify.shop_domain`, `shopify.access_token` (legacy fallback), **`shopify.client_id`**=`aa69269b292cd550495bc2aa463625bc`,
**`shopify.client_secret`**=`shpss_…` (PUI). Source of truth for the creds: `/home/shopify/services/shopify-api.sh`
(read-only repo). After deploy, ALL Shopify auth for this tenant uses client_credentials minting; the
static token becomes unused but harmless. The minted token's 177 scopes are a superset of the old 41 → no regression.

### New connector endpoint
`GET /v1/shopify/disputes` — `_fetch_disputes` over `/shopify_payments/disputes.json`, PII-free projection
(id, order_id, amount, currency, status, type, reason, dates). Needs the disputes scope (now obtainable).

---

## §6 — DEPLOY (gated; do in this order)

1. **ERPNext (Pascucci CoA):** create `Shopify Disputed Funds - PUI` (Receivable / Current Asset) and
   `Chargeback Losses - PUI` (Expense). The dispute payout `131619618986` cannot export until the first
   exists; the other 3 payouts use only pre-existing accounts.
2. **Build + restart** `core-api` and `connector-shopify` (and `jobs` once the poller lands). On core-api
   start, `alembic upgrade head` runs — only relevant once migration `0036` is added.
3. **Re-sync** the 4 payouts (connector `/v1/sync`) so `raw_payload.transactions` carry `adjustment_reason`.
   (Already present today — they were minted before, but re-sync is safe.)
4. **Verify live**: `POST /v1/recon/shopify/payouts/{id}/preview` for each of the 4 → reconciles=True.
5. **Categorize** the 4 payouts.

## §7 — Gotchas / invariants
- Containers run **baked images**, NOT host mounts. Editing host files does nothing until rebuild. To
  dry-run new code without deploy: `docker cp` edited files over `/app/src/...` (no `--reload`, so the
  running uvicorn keeps old code in memory), run a one-shot `docker exec python`, then `docker cp` the
  baked originals back. Baked backups were at `/tmp/baked_recon.py`, `/tmp/baked_pc.py` during this work.
- `shopify_routing_key` in core-api MUST stay in sync with any connector-side key derivation.
- Never lower the default-refuse: an unknown `adjustment:<reason>` must 412 + show in the digest, never
  silently bucket. That is the guardrail that surfaced this whole issue.
- `fee_mapping` is keyed by cost_center STRING (its type annotation says tuple — pre-existing, leave it).
  `special_fee_mapping` is keyed by `(account, cost_center)`.
- Token: do NOT store a raw minted `shpat_` as the static token — it expires in 24h. Use client creds.