Compare commits

..

19 Commits
v1.1 ... v2

Author SHA1 Message Date
b28662d3b4 Release 2.0.0 — explicit monorepo_url, drop first-target-host inference
Breaking change. The monorepo's gitea host was previously inferred from the
first target's subrepo_url, silently coupling the two and breaking on any
multi-host setup (monorepo on host A, target on host B). v2.0.0 replaces the
inference with a required josh.monorepo_url field that mirrors subrepo_url —
same parsing, same SSH/HTTPS auth options, same shape.

- schema_version: 2 is now required; v1 configs are rejected with a clear migration error
- josh.monorepo_url is required; josh.monorepo_auth is optional (default https)
- mono_auth_url() in lib/auth.sh mirrors subrepo_auth_url()
- MONOREPO_API derives from monorepo_url's host instead of .[0].gitea_host;
  env-var override preserved as escape hatch
- initial_import() and force-push call mono_auth_url() instead of building URLs inline
- ADR-012 documents the rationale; CHANGELOG includes migration snippet

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 06:31:21 +03:00
5e9b134168 Add --force flag to reverse sync
Bypasses the PR step and force-pushes the subrepo's state directly onto
the monorepo branch. Also skips the skip-dirty guard. Only valid with
--reverse. Returns status "forced".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 01:43:47 +03:00
1cb23a4411 Release 1.4.0
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-01 02:46:43 +03:00
479b592076 Add versioning contract: semver policy, breaking change sections, schema_version guard
- CHANGELOG: add ### Breaking Changes section to all versions (None for 1.1–1.3)
- README: add Versioning section with semver table and floating tag explanation
- schema/config-schema.json: add schema_version field (const: 1, optional)
- lib/config.sh: fail fast if schema_version is present and unsupported

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-01 00:56:33 +03:00
65cc8eaf8a Trim overengineered docs and ADR
Remove redundant step-by-step subsections from workflow guide (the
table already covers it), tighten ADR-011 alternatives, and remove
version pins that go stale.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 14:55:53 +03:00
4861179c6f Fix linearize fallback for already-applied merge commits
When cherry-pick -m 1 fails because the merge's changes were already
applied by a prior cherry-pick (empty result, clean working tree),
skip the commit instead of trying git-apply which also fails. Only
use the diff-from-first-parent fallback for real conflicts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 14:49:10 +03:00
c9b37f2477 Fix linearize fallback applying wrong diff (full tree instead of parent)
The fallback for failed cherry-picks used `git diff HEAD $sha` which
computes the diff between the linearize branch and the commit's full
tree. This captures ALL tree differences, not just the changes the
commit introduced — causing unrelated file deletions.

Fix: diff from the commit's own parent (`${sha}^` for regular commits,
`${sha}^1` for merge commits). Also detect and skip empty merges
(where cherry-pick -m 1 produces nothing because the changes were
already applied by a prior cherry-pick).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 14:41:04 +03:00
d3799187a1 Clean up linearize fallback output formatting
Suppress git stdout/stderr noise from cherry-pick, checkout, and
commit during linearization. Add delimiters and consistent indented
log lines so the output is easy to scan.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 14:33:42 +03:00
b4eaa47ef6 Fix reverse sync missing human commits when mono-filtered advances
--ancestry-path returns empty when mono-filtered/main has advanced
past the last forward sync (any monorepo commit causes josh to create
a new filtered SHA not in the subrepo's ancestry). Use git merge-base
to find the last connected point instead.

Also return skip-dirty (not skip) when trees differ but no human
commits found, preventing state from being updated on false skips
which would permanently lose unsynced commits.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 14:07:20 +03:00
d6f334b861 Add linearize fallback for reverse sync and workflow guide (v1.3)
When josh-proxy rejects a reverse sync push due to unmappable merge
commits, fall back to linearizing: cherry-pick regular commits
individually, squash only the merge commits via cherry-pick -m 1.

Also adds a recommended Git workflow section to the guide explaining
where cross-branch merges should happen (monorepo) vs feature work
(subrepo), and expands troubleshooting for the "josh rejected push"
error with root cause analysis and prevention advice.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 12:48:17 +03:00
8ab07b83ab Update docs, changelog, examples, and add ADRs for v1.2
- Add v1.1.0 and v1.2.0 changelog entries
- Add exclude field to config reference and example config
- Add ADRs documenting all major design decisions
- Fix step numbering in reverse_sync()
- Fix action.yml to copy VERSION file
- Add dist/ and .env to .gitignore
- Use refs/tags/ format for Nix flake tag refs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 21:28:40 +03:00
95b83bd538 Fix PR body newlines rendering as literal \n
Bash double-quoted strings don't interpret \n as newlines.
Use actual newlines in the pr_body strings instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 16:13:13 +03:00
ce53d3c1d2 Fix reconciliation parent order and add reverse sync tree check
- Swap parent order in reconcile_filter_change(): josh-filtered must
  be first parent so josh can follow first-parent traversal to map
  history back to the monorepo. Old subrepo history on parent 2.
- Add tree comparison in reverse_sync() before commit detection:
  if subrepo tree matches josh-filtered tree, skip immediately.
  Prevents false positive PRs after reconciliation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 15:11:31 +03:00
16257f25d7 Fix reverse sync false positive after filter reconciliation
Add --ancestry-path to git log in reverse_sync() to prevent old
subrepo history from leaking through reconciliation merge parents.
Without this, every old subrepo commit appears as a "human commit"
triggering a spurious 0-commit PR on the monorepo.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 14:19:56 +03:00
c0ddb887ff Fix filter reconciliation for pre-v1.2 state and unrelated histories
Three bugs found during first CI run after enabling :exclude:

- Derive old filter (:/subfolder) when state has no josh_filter stored
  (pre-v1.2 upgrade path)
- Detect unrelated histories in forward_sync() and fall back to
  reconcile_filter_change() instead of creating a useless conflict PR
- Skip state update on conflict result (prevents storing wrong filter
  and mono SHA that blocks retries)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 13:30:24 +03:00
22bd59a9d7 Auto-reconcile subrepo history when josh filter changes
When the exclude list changes, josh-proxy recomputes filtered history
with new SHAs, breaking common ancestry with the subrepo. Instead of
requiring a manual reset (force-push), forward sync now detects the
filter change and creates a reconciliation merge commit that connects
the old and new histories — no force-push, no re-clone needed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 10:40:08 +03:00
d7f8618b38 Use inline :exclude in josh-proxy URL instead of stored filter files
The :+ stored filter syntax doesn't work in josh-proxy URLs.
Inline :exclude[::p1,::p2] works directly — no files to generate
or commit, no extra dependencies.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 10:19:41 +03:00
5929585d6c Fix josh-proxy rejecting stored filter path with slash
Josh-proxy's parser treats "/" in :+ paths as a filter separator,
so :+.josh-filters/backend fails. Use flat naming at repo root:
.josh-filter-<target>.josh referenced as :+.josh-filter-<target>.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 09:47:16 +03:00
187a9ead14 Add file exclusion via josh stored filters (v1.2.0)
New `exclude` config field per target generates .josh-filters/<name>.josh
files with josh :exclude clauses. Josh-proxy applies exclusions at the
transport layer — excluded files never appear in the subrepo.

Preflight checks that generated filter files are committed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 22:45:13 +03:00
33 changed files with 1563 additions and 69 deletions

3
.gitignore vendored
View File

@@ -1,3 +1,4 @@
.claude/*local*
dist/
.env
result

View File

@@ -1,5 +1,131 @@
# Changelog
## 2.0.0
### Breaking Changes
- **`schema_version: 2` is now required.** v1 configs are rejected with a migration pointer. Add `schema_version: 2` at the top of `.josh-sync.yml`.
- **`josh.monorepo_url` is now required.** Set it to your monorepo's git clone URL — `git@host:org/repo.git` (SSH), `ssh://...`, or `https://host/org/repo.git`. Removes the silent inference of the gitea host from the first target's `subrepo_url`, which silently misbehaved when the monorepo and a target lived on different gitea instances. See [ADR-012](docs/adr/012-explicit-monorepo-url.md).
- **`MONOREPO_API` derivation changed.** The default is now `https://{monorepo_url's host}/api/v1/repos/{monorepo_path}`. The env-var override is preserved as an escape hatch for custom API endpoints.
#### Migration
```yaml
# Before (v1.x)
josh:
proxy_url: "https://josh.example.com"
monorepo_path: "org/monorepo"
# After (v2.0.0)
schema_version: 2
josh:
proxy_url: "https://josh.example.com"
monorepo_path: "org/monorepo"
monorepo_url: "git@gitea.example.com:org/monorepo.git" # required
monorepo_auth: "ssh" # optional, default "https"
```
If you previously set the `MONOREPO_API` env var to work around the first-target-host bug, you can remove it — the new derivation gets it right from `monorepo_url`.
### Features
- **Multi-host configs supported.** The monorepo and any subrepo may live on different gitea instances. Target ordering no longer affects how the monorepo's host is resolved.
- **SSH clone of the monorepo via `monorepo_auth: ssh`.** Uses the user's ssh-agent / `~/.ssh/config`; no token required for the clone itself. PR creation still uses `GITEA_TOKEN`.
- **Symmetric config shape.** Monorepo and subrepos now both declare a URL plus an auth method.
### Fixes
- First-target ordering no longer load-bearing on `MONOREPO_API` derivation (was `gitea_host=.[0].gitea_host` in v1.x).
## 1.5.0
### Breaking Changes
_None._
### Features
- **`--force` flag for reverse sync**: `josh-sync sync --reverse --force` bypasses
the PR step and force-pushes the subrepo's state directly onto the monorepo branch.
Also skips the `skip-dirty` guard. Only valid with `--reverse`. Returns status `forced`.
## 1.4.0
### Breaking Changes
_None._
### Features
- **`schema_version` config field**: New optional integer field in `.josh-sync.yml`. Defaults to `1` when absent. josh-sync fails fast with a clear error if an unsupported version is present — safe path for future config migrations without silent misinterpretation.
### Fixes
- **Linearize fallback applying wrong diff**: Fallback for failed `cherry-pick` was diffing from `HEAD` to the commit's full tree instead of from the commit's own parent, causing unrelated file deletions. Now diffs from the commit's own parent (`${sha}^` / `${sha}^1`).
- **Linearize fallback for already-applied merge commits**: When `cherry-pick -m 1` produces an empty result because a merge's changes were already applied by a prior cherry-pick, the commit is now skipped instead of attempting a failing `git apply`.
- **Linearize fallback output**: Suppressed git noise from cherry-pick, checkout, and commit during linearization; added delimiters and consistent indented log lines.
### Docs
- Workflow guide: removed redundant step-by-step subsections (table already covers them); tightened ADR-011 alternatives; removed stale version pins.
- README: added Versioning section with semver policy table and floating tag explanation.
## 1.3.0
### Breaking Changes
_None._
### Features
- **Linearize fallback for reverse sync**: When josh-proxy rejects a push due to unmappable merge commits, reverse sync automatically falls back to linearizing the history — cherry-picking regular commits individually and squashing only the problematic merge commits via `cherry-pick -m 1`. Preserves individual commit granularity while handling complex merge topologies that josh cannot map. PR body notes when linearization was used.
### Fixes
- **Reverse sync human commit detection**: Fixed `--ancestry-path` returning empty when `mono-filtered` has advanced past the last forward sync. Any monorepo commit (even outside the subfolder) causes josh to create a new filtered commit, breaking the ancestry path to subrepo HEAD. Now uses `git merge-base` to find the last connected point.
- **State not updated on false skip**: When trees differ but no human commits are found, reverse sync now returns `skip-dirty` instead of `skip`. State is not updated, so the system retries on the next run instead of silently dropping commits.
### Docs
- Expanded troubleshooting guide for "Josh rejected push" with root cause analysis and prevention advice.
- Added ADR-011: Linearize fallback for reverse sync.
- Added recommended Git workflow section to the guide.
## 1.2.0
### Breaking Changes
_None._
### Features
- **File exclusion**: `exclude` config field removes files/directories from the subrepo at the josh-proxy transport layer. Patterns are embedded inline in the josh-proxy URL using `:exclude[::pattern,...]` syntax — no extra files to generate or commit.
- **Filter change reconciliation**: When the josh filter changes (e.g., adding/removing exclude patterns), josh-sync automatically creates a reconciliation merge commit that connects old and new histories. No manual reset or force-push required.
- **Tree comparison guard**: Reverse sync now compares subrepo tree to josh-filtered tree before checking commit log. Skips immediately when trees are identical, avoiding false positives from reconciliation merge history.
- **Unrelated histories detection**: Forward sync detects when histories are unrelated (no common ancestor) and falls back to reconciliation instead of creating a useless conflict PR.
### Fixes
- Pre-v1.2 state compatibility: When upgrading from v1.0/v1.1 (no `josh_filter` stored in state), the old filter is derived from `subfolder` so reconciliation triggers correctly.
- Reconciliation merge parent order: Josh-filtered history is always first parent so josh-proxy can follow first-parent traversal back to the monorepo.
- Reverse sync `--ancestry-path` flag prevents old subrepo history from leaking through reconciliation merge parents.
- PR body `\n` now renders as actual newlines instead of literal text.
- Conflict result no longer updates sync state (added `continue` to skip state write).
- `action.yml` now copies VERSION file for correct `--version` output in CI.
- `.gitignore` now includes `dist/` and `.env`.
## 1.1.0
### Breaking Changes
_None._
### Features
- **`onboard` command**: Interactive, resumable workflow for importing existing subrepos into the monorepo. Walks through: prerequisites check, import (creates PRs), wait for merge, reset (pushes josh-filtered history). Checkpoint/resume at every step.
- **`migrate-pr` command**: Migrates open PRs from an archived subrepo to the new one. Supports interactive selection, `--all` flag, and specific PR numbers. Uses `git apply --3way` for resilient patch application.
- **Onboard state tracking**: Stored on the `josh-sync-state` branch at `<target>/onboard.json`. Tracks step progress, import PR numbers, reset branches, and migrated PRs.
## 1.0.0
Initial release. Extracted from [private-monorepo-example](https://code.itkan.io/pe/private-monorepo-example) into a standalone reusable library.

View File

@@ -9,19 +9,23 @@ Bidirectional monorepo ↔ subrepo sync via [josh-proxy](https://josh-project.gi
Create `.josh-sync.yml` in your monorepo root:
```yaml
schema_version: 2
josh:
proxy_url: "https://josh.example.com"
monorepo_path: "org/monorepo"
monorepo_url: "git@gitea.example.com:org/monorepo.git"
monorepo_auth: "ssh" # optional, default "https"
targets:
- name: "billing"
subfolder: "services/billing"
josh_filter: ":/services/billing"
subrepo_url: "git@gitea.example.com:ext/billing.git"
subrepo_auth: "ssh"
branches:
main: main
forward_only: []
exclude: # files excluded from subrepo (optional)
- ".monorepo/"
bot:
name: "josh-sync-bot"
@@ -58,13 +62,27 @@ Run `josh-sync preflight` to validate your setup.
## Documentation
- **[Setup Guide](docs/guide.md)** — Step-by-step: prerequisites, importing existing subrepos, CI workflows, and troubleshooting
- **[Setup Guide](docs/guide.md)** — Step-by-step: prerequisites, importing existing subrepos, CI workflows, file exclusion, and troubleshooting
- **[Configuration Reference](docs/config-reference.md)** — Full `.josh-sync.yml` field documentation
- **[Architecture Decision Records](docs/adr/)** — Design rationale and trade-offs
- **[Changelog](CHANGELOG.md)** — Version history
## Versioning
josh-sync follows [semver](https://semver.org/):
| Bump | Meaning | Action required |
|------|---------|-----------------|
| **Patch** (1.x.**y**) | Bug fixes only | Safe to upgrade |
| **Minor** (1.**y**.0) | New optional config fields, new commands, new action inputs | Safe to upgrade; new features are opt-in |
| **Major** (**y**.0.0) | Removed or renamed config fields, CLI flags, action inputs, env vars, or state format | Read the **Breaking Changes** section in the changelog before upgrading |
CI workflows pin to a floating major tag (e.g. `@v1`). A breaking change bumps the major version and moves the tag — `@v1` workflows keep working until you explicitly update to `@v2`.
## CLI
```
josh-sync sync [--forward|--reverse] [--target NAME[,NAME]] [--branch BRANCH]
josh-sync sync [--forward|--reverse] [--force] [--target NAME[,NAME]] [--branch BRANCH]
josh-sync preflight
josh-sync import <target>
josh-sync reset <target>
@@ -78,13 +96,17 @@ josh-sync state reset <target> [branch]
## How It Works
- **Forward sync** (mono → subrepo): pushes directly if clean, creates conflict PR if not. Uses `--force-with-lease` for safety.
- **Reverse sync** (subrepo → mono): always creates a PR, never pushes directly.
- **Reverse sync** (subrepo → mono): creates a PR by default. Add `--force` to bypass the PR and push directly to the mono branch (destructive).
- **File exclusion**: `exclude` patterns are embedded inline in the josh-proxy URL. Excluded files exist only in the monorepo.
- **Filter reconciliation**: Changing the exclude list auto-creates a merge commit that connects old and new histories — no force-push needed.
- **Loop prevention**: `Josh-Sync-Origin:` git trailer filters out bot commits.
- **State tracking**: orphan branch `josh-sync-state` stores JSON per target/branch.
## Dependencies
`bash >=4`, `git`, `curl`, `jq`, `yq` ([mikefarah/yq](https://github.com/mikefarah/yq) v4+), `openssh`
`bash >=4`, `git`, `curl`, `jq`, `yq` ([mikefarah/yq](https://github.com/mikefarah/yq) v4+), `openssh`, `rsync`
> The Nix flake bundles all dependencies automatically.
## License

View File

@@ -1 +1 @@
1.1.0
2.0.0

View File

@@ -26,6 +26,7 @@ runs:
run: |
JOSH_DIR="$(mktemp -d)"
cp -r "${{ github.action_path }}/bin" "${{ github.action_path }}/lib" "${JOSH_DIR}/"
cp "${{ github.action_path }}/VERSION" "${JOSH_DIR}/" 2>/dev/null || true
chmod +x "${JOSH_DIR}/bin/josh-sync"
echo "${JOSH_DIR}/bin" >> "$GITHUB_PATH"
echo "JOSH_SYNC_ROOT=${JOSH_DIR}" >> "$GITHUB_ENV"

View File

@@ -88,6 +88,7 @@ Global flags:
Sync flags:
--forward Forward only (mono → subrepo)
--reverse Reverse only (subrepo → mono)
--force Force-push subrepo state to mono branch (--reverse only, skips PR)
--target NAME Filter to target(s) — comma-separated for multiple (env: JOSH_SYNC_TARGET)
--branch BRANCH Filter to one branch
@@ -108,6 +109,7 @@ cmd_sync() {
local filter_target="${JOSH_SYNC_TARGET:-}"
local filter_branch=""
local config_file=".josh-sync.yml"
local force_flag=false
while [ $# -gt 0 ]; do
case "$1" in
@@ -117,10 +119,18 @@ cmd_sync() {
--branch) filter_branch="$2"; shift 2 ;;
--config) config_file="$2"; shift 2 ;;
--debug) export JOSH_SYNC_DEBUG=1; shift ;;
--force) force_flag=true; shift ;;
*) die "Unknown flag: $1" ;;
esac
done
if [ "$force_flag" = true ] && [ "$direction" != "reverse" ]; then
die "--force requires --reverse"
fi
if [ "$force_flag" = true ]; then
export SYNC_REVERSE_FORCE=1
fi
parse_config "$config_file"
# Forward sync
@@ -208,13 +218,42 @@ _sync_direction() {
fi
fi
# Run sync
# Check for filter change (forward only — reverse uses same filter)
local result
if [ "$direction" = "forward" ]; then
result=$(forward_sync)
local prev_filter
prev_filter=$(echo "$state" | jq -r '.last_forward.josh_filter // empty')
# If no filter stored (pre-v1.2 state) but a previous sync exists,
# the old filter was the simple :/subfolder (before exclude was added)
if [ -z "$prev_filter" ]; then
local prev_mono_sha
prev_mono_sha=$(echo "$state" | jq -r '.last_forward.mono_sha // empty')
if [ -n "$prev_mono_sha" ]; then
local subfolder
subfolder=$(echo "$TARGET_JSON" | jq -r '.subfolder')
prev_filter=":/${subfolder}"
fi
fi
if [ -n "$prev_filter" ] && [ "$prev_filter" != "$JOSH_FILTER" ]; then
log "WARN" "Josh filter changed — reconciling histories"
log "INFO" "Old: ${prev_filter}"
log "INFO" "New: ${JOSH_FILTER}"
result=$(reconcile_filter_change)
else
result=$(forward_sync)
fi
else
result=$(reverse_sync)
fi
# If forward sync hit unrelated histories, fall back to reconciliation
if [ "$result" = "unrelated" ]; then
log "WARN" "Unrelated histories detected — falling back to filter reconciliation"
result=$(reconcile_filter_change)
log "INFO" "Reconciliation result: ${result}"
fi
log "INFO" "Result: ${result}"
# Handle warnings
@@ -224,11 +263,19 @@ _sync_direction() {
fi
if [ "$result" = "conflict" ]; then
echo "::warning::Target ${target_name}, branch ${branch}: merge conflict — PR created on subrepo"
continue
fi
if [ "$result" = "josh-rejected" ]; then
echo "::error::Target ${target_name}, branch ${branch}: josh rejected push — check proxy logs"
continue
fi
if [ "$result" = "skip-dirty" ]; then
log "WARN" "Trees differ but no human commits — skipping state update (will retry)"
continue
fi
if [ "$result" = "forced" ]; then
log "WARN" "Target ${target_name}, branch ${branch}: force-pushed subrepo directly to mono — DESTRUCTIVE"
fi
# Update state (only on success)
local new_state
@@ -240,8 +287,9 @@ _sync_direction() {
--arg s_sha "${subrepo_sha_now:-}" \
--arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
--arg status "$result" \
--arg filter "$JOSH_FILTER" \
--argjson prev "$state" \
'$prev + {last_forward: {mono_sha:$m_sha, subrepo_sha:$s_sha, timestamp:$ts, status:$status}}')
'$prev + {last_forward: {mono_sha:$m_sha, subrepo_sha:$s_sha, timestamp:$ts, status:$status, josh_filter:$filter}}')
else
local mono_sha_now
mono_sha_now=$(git rev-parse "origin/${branch}" 2>/dev/null || echo "")

View File

@@ -0,0 +1,42 @@
# ADR-001: Josh-proxy for Bidirectional Sync
**Status:** Accepted
**Date:** 2026-01
## Context
We need bidirectional sync between a monorepo and N external subrepos. Each subrepo corresponds to a subfolder in the monorepo. Developers on both sides should see a clean, complete git history — not synthetic commits or squashed blobs.
### Alternatives considered
1. **git subtree**: Built into git. `git subtree split` extracts a subfolder into a standalone repo. However, subtree split rewrites history on every run (O(n) on total commits), creating new SHAs each time. Bidirectional sync requires manual `subtree merge` with conflict-prone history grafting. No transport-layer filtering — all content must be fetched.
2. **git submodule**: Tracks external repos via `.gitmodules` pointer commits. Does not provide content-level integration — monorepo commits don't contain subrepo files directly. Developers must run `git submodule update`. Bidirectional sync is not a supported workflow.
3. **Custom diff-and-patch scripts**: Compute diffs between monorepo subfolder and subrepo, apply patches in both directions. Fragile with renames, binary files, and merge conflicts. Loses authorship and commit granularity.
4. **josh-proxy**: A git proxy that computes filtered views of repositories in real-time. Clients `git clone` through josh and receive a repo containing only the specified subfolder, with history rewritten to match. Josh maintains a persistent SHA mapping, so the same monorepo commit always produces the same filtered SHA. Bidirectional: pushing back through josh maps filtered commits to monorepo commits.
## Decision
Use josh-proxy as the transport layer for all sync operations.
## Consequences
**Positive:**
- Clean git history in both directions — no synthetic commits
- Deterministic SHA mapping — same monorepo state always produces same filtered SHA
- Bidirectional by design — push through josh maps back to monorepo
- Transport-layer filtering — content exclusion happens at clone/push time, not via generated files
- Supports any git hosting platform (Gitea, GitHub, GitLab) since it's a proxy
**Negative:**
- Requires running a josh-proxy instance (operational overhead)
- Josh-proxy is a Rust project with a smaller community than git-native tools
- Proxy must have network access to the monorepo's git server
- Josh's SHA mapping is opaque — debugging requires understanding josh internals
- First-parent traversal behavior must be respected in merge commits (see ADR-008)
**Risks:**
- Josh-proxy downtime blocks all sync operations
- Josh-proxy bugs could corrupt history mapping (mitigated by force-with-lease on forward, always-PR on reverse)

View File

@@ -0,0 +1,50 @@
# ADR-002: State Storage on Orphan Git Branch
**Status:** Accepted
**Date:** 2026-01
## Context
Josh-sync needs persistent state to track what has already been synced (last-synced commit SHAs, timestamps, status). This prevents re-syncing unchanged content and enables incremental operation. The state must survive CI runner teardown — runners are ephemeral containers.
### Alternatives considered
1. **File in the repo**: Commit a state JSON file to the monorepo. Every sync run creates a commit, polluting history. Race conditions when multiple sync jobs run concurrently.
2. **External database/KV store**: Redis, SQLite, or a cloud KV service. Adds an infrastructure dependency. Credentials and connectivity to manage.
3. **CI artifacts/cache**: Platform-specific (GitHub Actions cache, Gitea cache). Not portable across CI platforms. Expiry policies vary.
4. **Orphan git branch**: A branch with no parent relationship to the main history. Stores JSON files in a simple `<target>/<branch>.json` layout. Pushed to origin, so it survives runner teardown. No external dependencies — uses git itself.
## Decision
Store sync state as JSON files on an orphan branch (`josh-sync-state`) in the monorepo.
### Storage layout
```
origin/josh-sync-state/
<target>/<branch>.json # sync state per target/branch
<target>/onboard.json # onboard workflow state (v1.1+)
```
### Implementation
- `read_state()`: `git fetch origin josh-sync-state && git show origin/josh-sync-state:<key>.json`
- `write_state()`: Uses `git worktree` to check out the orphan branch in a temp directory, writes JSON, commits, and pushes. This avoids touching the main working tree.
## Consequences
**Positive:**
- Zero external dependencies — only git
- Portable across CI platforms (Gitea Actions, GitHub Actions, local)
- Human-readable JSON files — easy to inspect and debug
- Atomic updates via git commit + push
- Natural namespacing via directory structure
**Negative:**
- Concurrent writes can race (mitigated by concurrency groups in CI workflows)
- `git worktree` adds complexity to the write path
- State branch appears in `git branch -a` output (minor clutter)
- Push failures on the state branch are non-fatal (logged as warning, sync still succeeds)

View File

@@ -0,0 +1,33 @@
# ADR-003: Force-with-Lease for Forward Sync
**Status:** Accepted
**Date:** 2026-01
## Context
Forward sync pushes monorepo changes to the subrepo. If someone pushes directly to the subrepo between when josh-sync reads its HEAD and when josh-sync pushes, a naive `git push` would overwrite their work. A `git push --force` would be worse — it would silently destroy concurrent changes.
## Decision
Use `git push --force-with-lease=refs/heads/<branch>:<expected-sha>` for all forward sync pushes. The expected SHA is recorded at the start of the sync operation (the "lease").
### How it works
1. Record subrepo HEAD SHA before any operations: `subrepo_sha=$(subrepo_ls_remote "$branch")`
2. Perform merge of monorepo changes onto subrepo state
3. Push with explicit lease: `--force-with-lease=refs/heads/main:<subrepo_sha>`
4. If the subrepo HEAD changed since step 1, git rejects the push
5. Josh-sync reports `lease-rejected` and retries on the next run
## Consequences
**Positive:**
- Never overwrites concurrent changes — git atomically checks the expected SHA
- Explicit SHA lease (not just "current tracking ref") prevents stale-ref bugs
- Failed leases are retried on the next sync run — no data loss, just delay
- Works correctly with josh-proxy's SHA mapping
**Negative:**
- Lease-rejected means the sync run did work that gets discarded (clone, merge, etc.)
- Persistent lease failures indicate a concurrent push pattern that needs investigation
- Requires the `--force-with-lease` flag with explicit SHA — the shorthand form (`--force-with-lease` without `=`) is unsafe because it uses the local tracking ref, which may be stale

View File

@@ -0,0 +1,41 @@
# ADR-004: Always-PR Policy for Reverse Sync
**Status:** Accepted
**Date:** 2026-01
## Context
Reverse sync brings subrepo changes back into the monorepo. The monorepo is the source of truth and typically has CI checks, code review requirements, and branch protection rules. Pushing directly to the monorepo's main branch would bypass these safeguards.
### Alternatives considered
1. **Direct push**: Fast, but bypasses all review and CI. A bad subrepo commit could break the entire monorepo with no review gate.
2. **Always create a PR**: Pushes to a staging branch (`auto-sync/subrepo-<branch>-<timestamp>`), then creates a PR via API. Humans review and merge.
3. **Configurable per-target**: Let users choose direct push vs PR. Adds complexity and a dangerous default.
## Decision
Reverse sync always creates a PR on the monorepo. Never pushes directly to the target branch.
### Implementation
1. Push subrepo HEAD through josh-proxy to a staging branch: `git push -o "base=main" josh://... HEAD:refs/heads/auto-sync/subrepo-main-<ts>`
2. Create PR via Gitea/GitHub API targeting the monorepo's main branch
3. PR includes a review checklist: scoped to subfolder, no leaked credentials, CI passes
The `-o "base=main"` option tells josh-proxy which monorepo branch to map the push against.
## Consequences
**Positive:**
- All monorepo changes go through review — consistent with team workflow
- CI runs on the PR branch before merge
- Bad subrepo changes are caught before they affect the monorepo
- Audit trail via PR history
**Negative:**
- Reverse sync is not instant — requires human action to merge the PR
- Stale PRs accumulate if subrepo changes frequently but PRs aren't merged promptly
- Adds API dependency (needs token with PR creation scope)

View File

@@ -0,0 +1,52 @@
# ADR-005: Git Trailer for Loop Prevention
**Status:** Accepted
**Date:** 2026-01
## Context
Bidirectional sync creates an infinite loop risk: forward sync pushes commit A to the subrepo, reverse sync sees commit A as "new" and creates a PR back to the monorepo, forward sync sees the merged PR as "new" and pushes again, etc.
### Alternatives considered
1. **SHA tracking only**: Compare SHAs to skip already-synced content. Breaks when josh-proxy rewrites SHAs (which it always does for filtered views). The monorepo commit SHA and the filtered/subrepo commit SHA are never the same.
2. **Commit message prefix**: Add `[sync]` to bot commit messages. Fragile — humans might use the same prefix. Requires string matching on message content.
3. **Git trailer**: A structured key-value pair in the commit message body (after a blank line), following the `git interpret-trailers` convention. Format: `Key: value`. Machine-parseable, unlikely to be used by humans, and supported by `git log --grep`.
## Decision
All bot commits include a git trailer with a configurable key (default: `Josh-Sync-Origin`). Both sync directions filter out commits containing this trailer.
### Format
```
Sync from monorepo 2026-02-12T10:30:00Z
Josh-Sync-Origin: forward/main/2026-02-12T10:30:00Z
```
The trailer value encodes: direction, branch, and timestamp. This aids debugging but is not parsed by the loop filter — only the trailer key presence matters.
### Filtering
- **Reverse sync**: `git log --invert-grep --grep="^${BOT_TRAILER}:"` excludes all commits with the trailer
- **CI loop guard**: The composite action checks if HEAD commit has the trailer before running sync at all
### Configuration
The trailer key is set in `.josh-sync.yml` under `bot.trailer`. This allows multiple josh-sync instances (with different bots) to operate on the same repos without interfering.
## Consequences
**Positive:**
- Reliable loop prevention — trailer is part of the immutable commit object
- Configurable key avoids conflicts between multiple sync bots
- Human-readable — `git log` shows the trailer in commit messages
- CI loop guard prevents unnecessary sync runs entirely
**Negative:**
- Commits with manually-added trailers matching the key would be incorrectly filtered
- Trailer must be in the commit body (after blank line), not the subject line
- Squash-and-merge on PRs may lose the trailer if the platform doesn't preserve commit message body

View File

@@ -0,0 +1,55 @@
# ADR-006: Inline Exclude in Josh-Proxy URL
**Status:** Accepted
**Date:** 2026-02
## Context
Some files in a monorepo subfolder should not appear in the subrepo (e.g., monorepo-specific CI configs, internal tooling, secrets templates). We need a mechanism to exclude these files from sync.
### Alternatives considered
1. **`.josh-sync-exclude` file committed to the repo**: A gitignore-style file listing patterns. Requires generating and committing a file. Changes to the exclude list create commits. The file itself would need to be excluded from the subrepo (circular dependency).
2. **Post-clone file deletion**: Clone through josh, then `rm -rf` excluded paths before pushing. Fragile — deletions create diff noise. Doesn't work for reverse sync (excluded files would appear as "deleted" in the subrepo).
3. **Josh `:exclude` filter inline in the URL**: Josh-proxy supports `:exclude[::pattern1,::pattern2]` appended to the filter path. The exclusion happens at the transport layer — git objects for excluded files are never transferred. Works identically for clone (forward) and push (reverse).
4. **Separate josh filter file**: Generate a josh filter expression and store it somewhere. Adds state management complexity.
## Decision
Embed exclusion patterns inline in the josh-proxy URL using josh's native `:exclude` syntax. The `exclude` config field in `.josh-sync.yml` is transformed at config parse time into the josh filter string.
### Example
Config:
```yaml
exclude:
- ".monorepo/"
- "**/internal/"
```
Produces josh filter:
```
:/services/billing:exclude[::.monorepo/,::**/internal/]
```
### Implementation
The `parse_config()` function in `lib/config.sh` uses jq to conditionally append `:exclude[...]` to the josh filter when the `exclude` array is non-empty. The enriched filter is stored in `JOSH_SYNC_TARGETS` JSON and used everywhere via `$JOSH_FILTER`.
## Consequences
**Positive:**
- Zero committed files — exclusion is purely in the URL
- Transport-layer filtering — excluded content never leaves the git server
- Works identically for forward sync (clone), reverse sync (push), and reset
- Tree comparison (`skip` detection) works correctly since excluded files aren't in the filtered view
- Standard josh syntax — no custom invention
**Negative:**
- Josh's `:exclude` pattern syntax is limited (no negation, no regex — only glob-style patterns with `::` prefix)
- Long exclude lists make the URL unwieldy (though this is cosmetic — git handles long URLs fine)
- Changing the exclude list changes the josh filter, which changes all filtered SHAs (see ADR-007 for how this is handled)
- Debugging requires understanding josh's filter composition syntax

View File

@@ -0,0 +1,53 @@
# ADR-007: Reconciliation Merge for Filter Changes
**Status:** Accepted
**Date:** 2026-02
## Context
When the josh filter changes (e.g., adding exclude patterns), josh-proxy recomputes the entire filtered history with new SHAs. The subrepo's existing history (based on the old filter) shares no common ancestor with the new filtered history. A naive forward sync would see "unrelated histories" and fail.
### Alternatives considered
1. **Force-push to subrepo**: Replace subrepo history with the new filtered view (same as `josh-sync reset`). Destructive — all local clones become invalid, open PRs are orphaned, developers must re-clone.
2. **Cherry-pick new commits**: Identify commits that exist in the new filtered history but not the old, cherry-pick them onto the subrepo. Complex — the "same" commit has different SHAs in old vs new filtered history. No reliable way to match them.
3. **Reconciliation merge commit**: Create a merge commit on the subrepo that has both the new filtered HEAD and the old subrepo HEAD as parents, using the new filtered tree. This establishes shared ancestry without rewriting history.
## Decision
When josh-sync detects a filter change (stored filter in state differs from current `$JOSH_FILTER`), create a reconciliation merge commit using `git commit-tree`.
### How it works
1. Clone subrepo (has old history)
2. Fetch josh-proxy filtered view (has new history)
3. If trees are identical → skip (filter change had no effect on content)
4. Create merge commit: `git commit-tree <josh-tree> -p <josh-head> -p <subrepo-head>`
5. Push with `--force-with-lease`
The merge commit uses the josh-filtered tree (new content) and has two parents:
- **Parent 1**: josh-filtered HEAD (new filter history) — must be first (see ADR-008)
- **Parent 2**: subrepo HEAD (old filter history) — preserves old history as a side branch
### Detection
Filter change is detected by comparing the stored `josh_filter` in sync state with the current `$JOSH_FILTER`. For pre-v1.2 state (no filter stored), the old filter is derived as `:/<subfolder>`.
As a reactive fallback, `forward_sync()` also detects unrelated histories via `git merge-base` and falls back to reconciliation.
## Consequences
**Positive:**
- Non-destructive — old history is preserved as parent 2 of the merge
- Developers don't need to re-clone the subrepo
- Open PRs on the subrepo remain valid (they're based on commits that are still ancestors)
- Automatic — no manual intervention needed when changing exclude patterns
- Force-with-lease protects against concurrent changes during reconciliation
**Negative:**
- The merge commit is synthetic (created by bot, not a real merge of concurrent work)
- Parent ordering is critical — wrong order breaks josh's reverse mapping (see ADR-008)
- The reconciliation merge contains a bot trailer, so reverse sync correctly ignores it
- If the subrepo has diverged significantly (manual commits during filter change), the reconciliation merge may produce unexpected tree content (uses josh-filtered tree unconditionally)

View File

@@ -0,0 +1,42 @@
# ADR-008: First-Parent Ordering in Reconciliation Merges
**Status:** Accepted
**Date:** 2026-02
## Context
Josh-proxy uses **first-parent traversal** when mapping subrepo history back to the monorepo. When you push a commit through josh-proxy, josh walks the first-parent chain to find a commit it can map to a monorepo commit. If the first parent leads to unmappable history, josh cannot reconstruct the monorepo-side branch correctly.
This became critical when the reconciliation merge (ADR-007) initially had the wrong parent order: old subrepo history as parent 1, josh-filtered as parent 2. Josh followed parent 1, couldn't find any mappable commit, and created a monorepo branch containing only the subrepo subfolder content — effectively deleting 1280 files from the rest of the monorepo.
## Decision
In reconciliation merge commits, the josh-filtered HEAD **must be parent 1** (first parent). The old subrepo HEAD is parent 2.
```bash
git commit-tree "$josh_tree" \
-p "$josh_head" \ # parent 1: josh-filtered — josh follows this
-p "$subrepo_head" \ # parent 2: old history — side branch, ignored by josh
-m "..."
```
### Why this is safe
- The old subrepo HEAD (`subrepo_head`) is still an ancestor of the merge commit regardless of parent order — push succeeds either way
- `--ancestry-path` in reverse sync still follows `B → M → C` regardless of parent order (it traces all paths, not just first-parent)
- Josh follows first-parent and finds the josh-filtered commit, which maps cleanly back to the monorepo
## Consequences
**Positive:**
- Josh can map the reconciliation merge back to the monorepo correctly
- Reverse sync through josh produces correct diffs (only subrepo-scoped changes)
- `git log --first-parent` on the subrepo shows the clean josh-filtered lineage
**Negative:**
- This is a subtle invariant — future changes to merge commit creation must preserve parent order
- The constraint is undocumented in josh-proxy's own documentation (discovered empirically)
- No automated test can verify this without a running josh-proxy instance
**Lesson learned:**
Parent order in `git commit-tree -p` is not cosmetic. For tools that rely on first-parent traversal (josh-proxy, `git log --first-parent`), parent 1 must be the "mainline" that the tool should follow.

View File

@@ -0,0 +1,53 @@
# ADR-009: Tree Comparison as Sync Skip Guard
**Status:** Accepted
**Date:** 2026-02
## Context
Both forward and reverse sync need to detect "nothing to do" quickly. The primary mechanism is SHA comparison against stored state (last-synced SHA). However, this misses cases where:
- State is reset or lost
- Reconciliation merges change SHAs without changing content
- Multiple sync runs overlap
Additionally, reverse sync originally relied on `git log <base>..HEAD` to find new commits. After a reconciliation merge, the `..` range can leak old subrepo history through the merge's second parent, creating false positives.
## Decision
Add tree-level comparison as an early skip guard in both forward and reverse sync. Compare the git tree objects (which represent directory content, not commit history) to determine if there's actually any content difference.
### Forward sync
```bash
mono_tree=$(git rev-parse 'HEAD^{tree}')
subrepo_tree=$(git rev-parse "subrepo/${branch}^{tree}")
[ "$mono_tree" = "$subrepo_tree" ] && echo "skip"
```
### Reverse sync
```bash
subrepo_tree=$(git rev-parse "HEAD^{tree}")
josh_tree=$(git rev-parse "mono-filtered/${branch}^{tree}")
[ "$subrepo_tree" = "$josh_tree" ] && echo "skip"
```
Tree comparison happens **before** commit log analysis. If trees are identical, there is definitionally nothing to sync, regardless of what the commit history looks like.
### Combined with `--ancestry-path`
For reverse sync, even when trees differ, `git log --ancestry-path` restricts the commit range to the direct lineage between the two endpoints. This prevents old history from leaking through reconciliation merge parents.
## Consequences
**Positive:**
- Eliminates false positives from reconciliation merges (trees are identical after reconciliation)
- Fast — tree SHA comparison is O(1), no content traversal
- Correct by definition — if trees match, content is identical
- Defense in depth — works even when state tracking has gaps
**Negative:**
- Tree comparison alone doesn't tell you *which* commits are new (still need `git log` for PR descriptions)
- Adds an extra `git rev-parse` call per sync direction (negligible cost)
- Cannot detect file-mode-only changes if josh normalizes modes (theoretical edge case)

View File

@@ -0,0 +1,76 @@
# ADR-010: Onboard Workflow with Checkpoint/Resume
**Status:** Accepted
**Date:** 2026-02
## Context
Onboarding an existing subrepo into the monorepo is a multi-step process that involves human interaction (renaming repos, merging PRs). The full flow is:
1. Prerequisites: rename existing repo, create new empty repo
2. Import: copy subrepo content into monorepo, create import PR(s)
3. Wait: human merges the import PR(s)
4. Reset: force-push josh-filtered history to the new empty repo
5. (Optional) Migrate open PRs from archived repo
Each step can fail or be interrupted. The process may span hours or days (waiting for PR review). If interrupted, restarting from scratch wastes work and can create duplicate PRs.
### Alternatives considered
1. **Single-shot script**: Run all steps in sequence. If interrupted, must restart from scratch. Duplicate PRs if import step is re-run.
2. **Manual step-by-step commands**: `import`, then manually run `reset`. Simple but error-prone — users may forget steps or run them out of order.
3. **Checkpoint/resume with persistent state**: Track the current step and intermediate results (PR numbers, reset branches) in persistent state. On re-run, resume from the last completed step.
## Decision
Implement `josh-sync onboard` as a checkpoint/resume workflow with state stored on the `josh-sync-state` branch at `<target>/onboard.json`.
### State machine
```
start → importing → waiting-for-merge → resetting → complete
```
Each transition is persisted before proceeding. Re-running `josh-sync onboard <target>` reads the current step and resumes.
### State schema
```json
{
"step": "waiting-for-merge",
"archived_api": "https://host/api/v1/repos/org/repo-archived",
"archived_url": "git@host:org/repo-archived.git",
"archived_auth": "ssh",
"import_prs": { "main": 42 },
"reset_branches": ["main"],
"migrated_prs": [
{ "old_number": 5, "new_number": 12, "title": "Fix login" }
],
"timestamp": "2026-02-10T14:30:00Z"
}
```
### Per-branch progress
Import and reset both iterate over branches. Progress is saved after each branch, so interruption mid-iteration resumes at the next unprocessed branch.
### PR migration
`josh-sync migrate-pr` is a separate command that reads onboard state (for the archived repo URL) and tracks migrated PRs. It uses `git apply --3way` for resilient patch application — the subrepo's content is identical after reset, so patches apply cleanly.
## Consequences
**Positive:**
- Safe to interrupt at any point — no duplicate work on resume
- Per-branch tracking prevents duplicate import PRs or redundant resets
- Archived repo URL stored in state — `migrate-pr` can operate independently
- `--restart` flag allows starting over if state is corrupted
- Human-friendly — prints instructions at each step
**Negative:**
- State management adds complexity (read/write onboard state, step validation)
- Interactive steps (`read -r`) are not suitable for fully automated pipelines
- Onboard state persists on the state branch even after completion (minor clutter)
- The step machine is linear — cannot skip steps or run them out of order

View File

@@ -0,0 +1,58 @@
# ADR-011: Linearize Fallback for Reverse Sync
**Status:** Accepted
**Date:** 2026-03
## Context
Josh-proxy rejects reverse sync pushes when the subrepo history contains merge commits whose parents it cannot map through the filter. This happens when:
1. A long-lived branch (e.g., `stage`) is merged into `main` via a merge commit
2. That branch contains auto-sync merge commits or criss-cross merges with `main`
3. Josh encounters the merge commit, tries to map both parents through the filter, and fails with a 500 error
This is a legitimate subrepo workflow — teams merge staging branches into main regularly. The sync tool should handle it without requiring teams to change their Git workflow.
### Alternatives considered
1. **Squash all commits into one**: Simple but destroys all commit granularity.
2. **Rewrite subrepo history**: Breaks josh's SHA mapping, forces all developers to re-clone.
3. **Linearize: cherry-pick regular commits, squash only merges**: Preserves individual commit granularity; only merge commits lose their multi-parent structure.
## Decision
When the direct push through josh-proxy fails, fall back to option 4: linearize the history by cherry-picking onto the josh-filtered base.
### How it works
1. Direct `git push` through josh-proxy is attempted first (existing behavior)
2. If it fails, create a temporary branch from `mono-filtered/<branch>`
3. Walk human commits (oldest-first, bot commits excluded) from the ancestry path:
- **Regular commits** (≤1 parent): `git cherry-pick <sha>` — preserves author, date, and message
- **Merge commits** (>1 parent): `git cherry-pick -m 1 <sha>` — applies the merge's diff relative to its first parent as a single commit
4. If cherry-pick conflicts (rare — usually due to ordering issues), fall back to `git diff | git apply` with the original author metadata
5. Push the linearized branch through josh-proxy
6. PR body includes a note explaining that merge commits were squashed
### What is preserved
- Individual non-merge commits (author, date, message, diff)
- The net effect of each merge commit (as a squashed single commit)
- The original commit list in the PR body for reference
### What is lost
- The multi-parent structure of merge commits (they become single-parent)
- The distinction between "changes introduced by the merge" vs "changes from each parent" — the merge is represented as its diff from first parent
## Consequences
**Positive:**
- Reverse sync no longer gets stuck on merge commits — handles the common case automatically
- Non-merge commits retain full granularity (no unnecessary squashing)
- Subrepo history is untouched — no rewriting, no broken sync relationship
- The PR body documents when linearization was used, so reviewers know
**Negative:**
- Merge commit semantics are lost on the monorepo side (they appear as regular commits)
- Adds complexity to the reverse sync path (two code paths: direct push and linearize fallback)

View File

@@ -0,0 +1,53 @@
# ADR-012: Explicit `monorepo_url` (drop first-target-host inference)
**Status:** Accepted
**Date:** 2026-04
## Context
josh-sync v1.x had no first-class field for the monorepo's git location. To call the gitea PR API and to clone the monorepo for `initial_import` / force-push, the script derived the gitea host from the **first target's `subrepo_url`**:
```bash
# lib/config.sh (v1.x)
gitea_host=$(echo "$JOSH_SYNC_TARGETS" | jq -r '.[0].gitea_host')
export MONOREPO_API="${MONOREPO_API:-https://${gitea_host}/api/v1/repos/${MONOREPO_PATH}}"
```
This worked silently for every existing config — studio, kinanah/stores, itkan/sdlc — because in each, target 0's host happens to match the monorepo's host. The portal monorepo broke the assumption: monorepo on `code.atome.ac:atome-portal/portal`, single target `bff` on `code.itkan.io:sdlc/odoo_bff_core`. The derivation produced `https://code.itkan.io/atome-portal/portal.git` (host borrowed from the wrong target), giving a 404 on `josh-sync import`.
Two issues stack:
1. **Implicit coupling**: target ordering becomes load-bearing in a non-obvious way. The `MONOREPO_API` env var was the only escape hatch, hidden in shell init.
2. **Asymmetry**: `subrepo_url` is declared explicitly per target with full SSH/HTTPS support; the monorepo had only a `monorepo_path` and was always cloned via HTTPS+token. Local users with SSH keys for the monorepo still had to provide a token.
### Alternatives considered
1. **Keep deriving, document the env override loudly.** Cheapest. Leaves the booby trap in place — still fails closed for any future cross-host config.
2. **Add `josh.gitea_host` only.** Fixes the host issue without enabling SSH cloning of the monorepo. Half a fix, and asymmetric with how subrepos are declared.
3. **Add `josh.monorepo_url` (full URL), mirror `subrepo_url` exactly.** Removes the inference; enables SSH clone; symmetric with subrepos. Breaking — every existing config needs to add the field.
## Decision
Add a required `josh.monorepo_url` field plus an optional `josh.monorepo_auth` (default `"https"`). Mirror the `subrepo_url` parsing and `subrepo_auth_url()` helper exactly:
- `parse_config` extracts `MONOREPO_GITEA_HOST` and `MONOREPO_REPO_PATH` from `monorepo_url` using the same jq regex as `subrepo_url`.
- `MONOREPO_API` derives from `MONOREPO_GITEA_HOST` instead of `.[0].gitea_host`. The env-var override stays as a real escape hatch (custom API endpoints).
- `mono_auth_url()` in `lib/auth.sh` mirrors `subrepo_auth_url()`: SSH returns the bare URL, HTTPS injects `${BOT_USER}:${GITEA_TOKEN}@` (auto-converting `git@host:org/repo.git` to `https://host/org/repo.git` first).
- `initial_import()` and the force-push path in `lib/sync.sh` call `mono_auth_url()` instead of constructing the URL inline from `MONOREPO_API`.
The change is shipped as **v2.0.0** with `schema_version: 2` required. v1 configs are rejected with a migration pointer; configs missing `monorepo_url` are rejected with the field name and an example.
## Consequences
**Positive:**
- No more first-target-ordering coupling. Add or reorder targets freely.
- Multi-host setups (monorepo on host A, subrepo on host B) are first-class.
- SSH clone of the monorepo works without a token (the user's ssh-agent / `~/.ssh/config` resolves the key). PR creation still needs `GITEA_TOKEN`.
- Symmetric config: monorepo and subrepos both declare a URL + auth method.
**Negative:**
- Breaking change. Every existing config must add `schema_version: 2` and `josh.monorepo_url`. Mitigated by the validation `die`s pointing at the field name and changelog.
- Two related-but-distinct fields (`monorepo_path` for the josh-proxy filter; `monorepo_url` for the gitea clone) — kept separate because josh-proxy's path may not always equal the gitea path in custom setups.
**Risks:**
- Users may set `monorepo_url` to a host that doesn't match their `proxy_url`'s upstream. Detection is out of scope for this ADR; a follow-up `josh-sync preflight` check could compare hosts and warn.

20
docs/adr/README.md Normal file
View File

@@ -0,0 +1,20 @@
# Architecture Decision Records
This directory contains Architecture Decision Records (ADRs) for josh-sync. Each ADR documents a significant design decision, its context, the alternatives considered, and the rationale for the chosen approach.
## Index
| ADR | Title | Status |
|-----|-------|--------|
| [001](001-josh-proxy-for-sync.md) | Josh-proxy for bidirectional sync | Accepted |
| [002](002-state-on-orphan-branch.md) | State storage on orphan git branch | Accepted |
| [003](003-force-with-lease-forward.md) | Force-with-lease for forward sync | Accepted |
| [004](004-always-pr-reverse.md) | Always-PR policy for reverse sync | Accepted |
| [005](005-git-trailer-loop-prevention.md) | Git trailer for loop prevention | Accepted |
| [006](006-inline-exclude-filter.md) | Inline exclude in josh-proxy URL | Accepted |
| [007](007-reconciliation-merge.md) | Reconciliation merge for filter changes | Accepted |
| [008](008-first-parent-ordering.md) | First-parent ordering in reconciliation merges | Accepted |
| [009](009-tree-comparison-guard.md) | Tree comparison as sync skip guard | Accepted |
| [010](010-onboard-checkpoint-resume.md) | Onboard workflow with checkpoint/resume | Accepted |
| [011](011-linearize-fallback-reverse.md) | Linearize fallback for reverse sync | Accepted |
| [012](012-explicit-monorepo-url.md) | Explicit `monorepo_url` (drop first-target-host inference) | Accepted |

View File

@@ -5,17 +5,26 @@ Full reference for `.josh-sync.yml` fields and environment variables.
## `.josh-sync.yml` Structure
```yaml
josh: # josh-proxy settings (required)
targets: # sync targets (required, at least 1)
bot: # bot identity for sync commits (required)
schema_version: 2 # required since v2.0.0
josh: # josh-proxy + monorepo settings (required)
targets: # sync targets (required, at least 1)
bot: # bot identity for sync commits (required)
```
## `schema_version`
| Type | Required | Description |
|------|----------|-------------|
| integer | Yes | Must be `2` for josh-sync v2.x configs. v1 configs are rejected; see [Changelog v2.0.0](../CHANGELOG.md#200) for migration. |
## `josh` Section
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `proxy_url` | string | Yes | Josh-proxy URL, no trailing slash. Must start with `http://` or `https://`. |
| `monorepo_path` | string | Yes | Repository path as josh-proxy sees it (e.g., `org/monorepo`). |
| `monorepo_url` | string | Yes | Monorepo git clone URL — `git@host:org/repo.git` (SSH), `ssh://...`, or `https://host/org/repo.git`. The gitea host is derived from this URL and used for the PR API and direct clones, decoupled from any target's host. |
| `monorepo_auth` | string | No (default `"https"`) | Clone auth method: `"https"` (injects `${SYNC_BOT_USER}:${SYNC_BOT_TOKEN}` into the URL) or `"ssh"` (uses the user's ssh-agent / `~/.ssh/config`). PR creation always uses `GITEA_TOKEN` regardless. |
## `targets[]` Section
@@ -32,6 +41,7 @@ Each target maps a monorepo subfolder to an external subrepo.
| `subrepo_ssh_key_var` | string | No | `"SUBREPO_SSH_KEY"` | Name of the env var holding the SSH private key for this target. |
| `branches` | object | Yes | — | Branch mapping: `mono_branch: subrepo_branch`. Each key-value pair syncs those branches bidirectionally. |
| `forward_only` | string[] | No | `[]` | Branches that only sync mono → subrepo, never reverse. |
| `exclude` | string[] | No | `[]` | File/directory patterns to exclude from sync via josh `:exclude` filter. Excluded files exist only in the monorepo, never in the subrepo. See [Excluding Files](guide.md#excluding-files-from-sync). |
## `bot` Section
@@ -72,7 +82,7 @@ targets:
| `JOSH_SYNC_TARGET` | Restrict sync to a single target | All targets |
| `JOSH_SYNC_STATE_BRANCH` | Name of the orphan branch for state storage | `josh-sync-state` |
| `JOSH_SYNC_DEBUG` | Enable verbose logging (`1` to enable) | `0` |
| `MONOREPO_API` | Override monorepo API URL | Auto-derived from first target's host |
| `MONOREPO_API` | Override the monorepo gitea PR API URL. Escape hatch for custom API endpoints; rarely needed. | `https://{monorepo_url's host}/api/v1/repos/{monorepo_path}` |
## JSON Schema

View File

@@ -78,9 +78,13 @@ git ls-remote https://josh.example.com/org/monorepo.git HEAD
Create `.josh-sync.yml` at the monorepo root. Each target maps a monorepo subfolder to an external subrepo:
```yaml
schema_version: 2
josh:
proxy_url: "https://josh.example.com" # josh-proxy URL (no trailing slash)
monorepo_path: "org/monorepo" # repo path as josh sees it
monorepo_path: "org/monorepo" # repo path as josh-proxy sees it
monorepo_url: "git@gitea.example.com:org/monorepo.git" # monorepo git clone URL
monorepo_auth: "ssh" # optional, default "https"
targets:
- name: "billing" # unique identifier
@@ -91,6 +95,9 @@ targets:
branches:
main: main # mono_branch: subrepo_branch
forward_only: []
exclude: # files excluded from subrepo (optional)
- ".monorepo/" # monorepo-only config dir
- "**/internal/" # internal dirs at any depth
- name: "auth"
subfolder: "services/auth"
@@ -183,7 +190,7 @@ To pin to a specific version, use a tag ref in `devenv.yaml`:
```yaml
josh-sync:
url: git+https://your-gitea.example.com/org/josh-sync?ref=v1.1
url: git+https://your-gitea.example.com/org/josh-sync?ref=refs/tags/v1.2
flake: true
```
@@ -515,6 +522,97 @@ Bot commits include a git trailer like `Josh-Sync-Origin: forward/main/2024-02-1
Sync state is stored as JSON files on an orphan branch (`josh-sync-state`), one file per target/branch. This tracks the last-synced commit SHAs and timestamps to avoid re-syncing the same changes.
## Recommended Git Workflow
Josh-proxy maps commits through a filter per-branch. It handles linear history and simple merges (short-lived feature branches) without issues. However, it **cannot map merge commits whose parents were created on the subrepo side** — because those commits were never pushed through josh and have no monorepo-side mapping.
This means cross-branch merges (e.g., `stage``main`) must happen on the monorepo side, where josh can filter the result cleanly.
### The rule
**The monorepo owns branch topology. The subrepo owns feature development.**
### What to do where
| Action | Where to do it | Why |
|--------|---------------|-----|
| Feature branch → `main` | **Subrepo** (PR, any merge strategy) | Short-lived branch with clean lineage — josh handles it |
| `stage``main` (promotion) | **Monorepo** | Cross-branch merge — forward sync propagates the result to both subrepo branches |
| `main``stage` (catch-up) | **Monorepo** | Same reason — avoids criss-cross merge history on subrepo |
| Hotfix to `main` | **Either side** | Single commit or small PR — works everywhere |
| Config/CI changes (monorepo-only) | **Monorepo** | Not synced to subrepo (use `exclude` for monorepo-only files) |
### What to avoid
- **Don't merge `stage` into `main` on the subrepo with a merge commit.** The merge parents include commits created on the subrepo side (forward sync merges, criss-cross merges) that josh has no mapping for. Josh rejects the push with a 500 error.
- **Don't merge `main` into `stage` on the subrepo.** Creates criss-cross merge history that causes the same josh mapping failure when `stage` is later merged back.
- **Don't rebase synced branches on the subrepo.** This rewrites commit SHAs that josh has already mapped, breaking the sync relationship.
### If you must merge cross-branch on the subrepo
Use a **squash merge**. A squash merge produces a single commit with one parent — josh can always map it. You lose the individual commit history on the target branch, but the sync goes through cleanly.
As a safety net, josh-sync automatically falls back to linearizing the history when josh rejects a push — cherry-picking regular commits individually and squashing only the problematic merge commits. See the [troubleshooting section](#josh-rejected-push-reverse-sync) and [ADR-011](adr/011-linearize-fallback-reverse.md).
## Excluding Files from Sync
Some files in the monorepo subfolder may not belong in the subrepo (e.g., monorepo-specific CI configs, internal tooling). The `exclude` config field removes these at the josh-proxy layer — excluded files never appear in the subrepo.
### Configuration
Add an `exclude` list to any target:
```yaml
targets:
- name: "billing"
subfolder: "services/billing"
subrepo_url: "git@host:org/billing.git"
exclude:
- ".monorepo/" # directory at subfolder root
- "**/internal/" # directory at any depth
- "*.secret" # files by extension
branches:
main: main
```
### How it works
When `exclude` is present, josh-sync appends an inline `:exclude` filter to the josh-proxy URL. For the example above, the josh filter becomes:
```
:/services/billing:exclude[::.monorepo/,::**/internal/,::*.secret]
```
Josh-proxy applies this filter at the transport layer — no extra files to generate or commit. This means:
- **Forward sync**: the filtered clone already excludes the files
- **Reverse sync**: pushes through josh also respect the exclusion
- **Reset**: the subrepo history never contains excluded files
- **Tree comparison**: `skip` detection works correctly (excluded files are not in the diff)
### Pattern syntax
Josh uses `::` patterns inside `:exclude[...]`:
| Pattern | Matches |
|---------|---------|
| `dir/` | Directory at subfolder root |
| `file` | File at subfolder root |
| `**/dir/` | Directory at any depth |
| `**/file` | File at any depth |
| `*.ext` | Glob pattern (single `*` only) |
### Setup
1. Add `exclude` to the target in `.josh-sync.yml`
2. Run `josh-sync preflight` to verify the filter works
3. Forward sync will now exclude the specified files
No extra files to generate or commit — the exclusion is embedded directly in the josh-proxy URL.
### Changing the exclude list
You can safely add or remove patterns from `exclude` at any time. When josh-sync detects that the filter has changed since the last sync, it automatically creates a reconciliation merge commit on the subrepo that connects the old and new histories — no manual reset or force-push required. Developers do not need to re-clone the subrepo.
## Adding a New Target
To add a new subrepo after initial setup:
@@ -554,7 +652,35 @@ Normal: the subrepo changed while sync was running. The next sync run will pick
### "Josh rejected push" (reverse sync)
Josh-proxy couldn't map the push back to the monorepo. Check josh-proxy logs, verify the josh filter is correct. May indicate a history divergence — consider running `josh-sync reset <target>`.
Josh-proxy couldn't map the push back to the monorepo. This has two common causes:
#### Merge commits with unmappable parents
**Symptom:** Josh returns `500 Internal Server Error` with a message like:
```
rejecting merge with 2 parents:
"Merge pull request 'stage' (#30) from stage into main" (c4fa3c9...)
1) "Merge branch 'auto-sync/import-...'" (4bf8704...)
2) "Merge branch 'main' into stage" (d021654...)
```
**Cause:** The subrepo has a merge commit whose parents josh-proxy cannot trace through its filter. This typically happens when:
- A long-lived branch (e.g., `stage`) is merged into `main` via a merge commit (not squash)
- That branch contains auto-sync merge commits or other history that doesn't exist in josh's filtered view
- Someone merges `main` into a feature/staging branch and then merges it back — the criss-cross parents confuse josh's mapping
**Automatic handling:** josh-sync automatically falls back to linearizing the history when the direct push fails. Regular commits are cherry-picked individually (preserving authorship and messages), while merge commits are squashed into single commits via `cherry-pick -m 1`. The PR notes when this fallback was used. See [ADR-011](adr/011-linearize-fallback-reverse.md).
**Prevention:** In josh-synced subrepos, prefer **squash merges** when merging long-lived branches (stage, develop) into the synced branch. Squash merges produce a single commit with no merge parents, which josh can always map. Regular feature branch merges (short-lived, no auto-sync history) are usually fine.
**Manual resolution (if automatic fallback also fails):** This indicates a more fundamental history issue. Options:
1. Cherry-pick the desired changes manually onto a clean branch and push through josh
2. Run `josh-sync reset <target>` to re-establish history (destructive — all subrepo clones must re-fetch)
#### Filter or path mismatch
Josh-proxy couldn't map the push due to an incorrect filter. Check josh-proxy logs, verify the `josh_filter` or `subfolder` in `.josh-sync.yml` is correct, and ensure the subfolder exists in the monorepo.
### Import PR shows "No changes"

View File

@@ -5,12 +5,12 @@
# In devenv.yaml:
# inputs:
# josh-sync:
# url: github:org/josh-sync/v1.0.0
# url: git+https://your-gitea.example.com/org/josh-sync?ref=refs/tags/v1.2
# flake: true
#
# Or in flake.nix:
# inputs.josh-sync = {
# url = "github:org/josh-sync/v1.0.0";
# url = "git+https://your-gitea.example.com/org/josh-sync?ref=refs/tags/v1.2";
# inputs.nixpkgs.follows = "nixpkgs";
# };
@@ -21,14 +21,16 @@
# josh-sync CLI is now available in the shell.
# Commands:
# josh-sync sync --forward Forward sync (mono → subrepo)
# josh-sync sync --reverse Reverse sync (subrepo → mono)
# josh-sync preflight Validate config and connectivity
# josh-sync import <target> Initial import from subrepo
# josh-sync reset <target> Reset subrepo to josh-filtered view
# josh-sync status Show target config and sync state
# josh-sync state show <t> [b] Show state JSON
# josh-sync state reset <t> [b] Reset state
# josh-sync sync --forward Forward sync (mono → subrepo)
# josh-sync sync --reverse Reverse sync (subrepo → mono)
# josh-sync preflight Validate config and connectivity
# josh-sync import <target> Initial import from subrepo
# josh-sync reset <target> Reset subrepo to josh-filtered view
# josh-sync onboard <target> Interactive import + reset workflow
# josh-sync migrate-pr <target> Migrate PRs from archived repo
# josh-sync status Show target config and sync state
# josh-sync state show <t> [b] Show state JSON
# josh-sync state reset <t> [b] Reset state
enterShell = ''
echo "Josh Sync available run 'josh-sync --help' for commands"

View File

@@ -62,7 +62,7 @@ jobs:
done | sort -u | paste -sd ',' -)
echo "targets=${TARGETS}" >> "$GITHUB_OUTPUT"
- uses: https://your-gitea.example.com/org/josh-sync@v1
- uses: https://your-gitea.example.com/org/josh-sync@v1.2
with:
direction: forward
target: ${{ github.event.inputs.target || steps.detect.outputs.targets }}

View File

@@ -1,26 +1,39 @@
# .josh-sync.yml — Multi-target configuration example
# Place this at the root of your monorepo.
# Required since v2.0.0. Bump alongside the josh-sync major version when
# breaking config changes ship.
schema_version: 2
josh:
# Your josh-proxy instance URL (no trailing slash)
proxy_url: "https://josh.example.com"
# Repo path as josh sees it (org/repo on your Gitea/GitHub)
# Repo path as josh-proxy sees it (org/repo)
monorepo_path: "org/monorepo"
# Monorepo git clone URL — git@host:org/repo.git (SSH) or https://host/org/repo.git.
# The gitea host is derived from this URL (used for the PR API and direct clones).
# The monorepo and any subrepo may live on different gitea hosts.
monorepo_url: "git@gitea.example.com:org/monorepo.git"
# Optional. Default "https". Use "ssh" to clone the monorepo via SSH (no token
# needed for the clone — PR creation still requires GITEA_TOKEN).
monorepo_auth: "ssh"
targets:
- name: "billing"
subfolder: "services/billing"
josh_filter: ":/services/billing"
# josh_filter auto-derived as ":/services/billing" if omitted
subrepo_url: "https://gitea.example.com/ext/billing.git"
subrepo_auth: "https"
branches:
main: main
develop: develop
forward_only: []
exclude: # files excluded from subrepo (optional)
- ".monorepo/" # directory at subfolder root
- "**/internal/" # directory at any depth
- name: "auth"
subfolder: "services/auth"
josh_filter: ":/services/auth"
subrepo_url: "git@gitea.example.com:ext/auth.git"
subrepo_auth: "ssh"
# Per-target credential override (reads from $AUTH_SSH_KEY instead of $SUBREPO_SSH_KEY)
@@ -31,7 +44,6 @@ targets:
- name: "shared-lib"
subfolder: "libs/shared"
josh_filter: ":/libs/shared"
subrepo_url: "https://gitea.example.com/ext/shared-lib.git"
branches:
main: main

View File

@@ -10,7 +10,7 @@ name: "Josh Sync ← Subrepo"
on:
schedule:
- cron: "0 1,7,13,19 * * *" # Every 6h, offset from forward
- cron: "0 1,7,13,19 * * *" # Every 6h, offset from forward
workflow_dispatch:
inputs:
target:
@@ -40,7 +40,7 @@ jobs:
curl -sL "https://github.com/mikefarah/yq/releases/download/v4.44.6/yq_linux_amd64" \
-o /usr/local/bin/yq && chmod +x /usr/local/bin/yq
- uses: https://your-gitea.example.com/org/josh-sync@v1
- uses: https://your-gitea.example.com/org/josh-sync@v1.2
with:
direction: reverse
target: ${{ github.event.inputs.target || '' }}

View File

@@ -2,7 +2,8 @@
# lib/auth.sh — Authenticated URLs, remote queries, and PR creation
#
# Requires: lib/core.sh and lib/config.sh sourced first
# Expects: JOSH_PROXY_URL, MONOREPO_PATH, BOT_USER, GITEA_TOKEN, JOSH_FILTER,
# Expects: JOSH_PROXY_URL, MONOREPO_PATH, MONOREPO_URL, MONOREPO_AUTH,
# BOT_USER, GITEA_TOKEN, JOSH_FILTER,
# SUBREPO_URL, SUBREPO_AUTH, SUBREPO_TOKEN (set by parse_config + load_target)
# ─── Josh-Proxy Auth URL ───────────────────────────────────────────
@@ -28,6 +29,27 @@ subrepo_auth_url() {
fi
}
# ─── Monorepo Auth URL ─────────────────────────────────────────────
# HTTPS: injects user:token into MONOREPO_URL (auto-converts git@ form to https)
# SSH: returns bare MONOREPO_URL (auth via user's ssh-agent / ~/.ssh/config,
# or via GIT_SSH_COMMAND if the caller wired one up for the monorepo host)
mono_auth_url() {
if [ "${MONOREPO_AUTH:-https}" = "ssh" ]; then
echo "$MONOREPO_URL"
else
local https_url="$MONOREPO_URL"
# Convert git@host:org/repo.git → https://host/org/repo.git so token injection works
if [[ "$https_url" == git@* ]]; then
https_url=$(echo "$https_url" | sed -E 's|^git@([^:]+):(.+)$|https://\1/\2|')
elif [[ "$https_url" == ssh://* ]]; then
https_url=$(echo "$https_url" | sed -E 's|^ssh://[^@]*@([^/]+)/(.+)$|https://\1/\2|')
fi
# shellcheck disable=SC2001
echo "$https_url" | sed "s|https://|https://${BOT_USER}:${GITEA_TOKEN}@|"
fi
}
# ─── Remote Queries ─────────────────────────────────────────────────
subrepo_ls_remote() {

View File

@@ -16,11 +16,21 @@ parse_config() {
local config_json
config_json=$(yq -o json "$config_file") || die "Failed to parse ${config_file}"
# Schema version check — required, must be 2 (v1 configs are no longer supported)
local schema_version
schema_version=$(echo "$config_json" | jq -r '.schema_version // empty')
[ -n "$schema_version" ] || die "schema_version missing in ${config_file} (required since v2.0.0 — set 'schema_version: 2' at the top of the file; see CHANGELOG.md for migration instructions)"
[ "$schema_version" = "2" ] || die "Unsupported schema_version ${schema_version} in ${config_file} (this version of josh-sync requires schema_version 2 — see CHANGELOG.md for migration instructions)"
# Export global values
export JOSH_PROXY_URL
JOSH_PROXY_URL=$(echo "$config_json" | jq -r '.josh.proxy_url')
export MONOREPO_PATH
MONOREPO_PATH=$(echo "$config_json" | jq -r '.josh.monorepo_path')
export MONOREPO_URL
MONOREPO_URL=$(echo "$config_json" | jq -r '.josh.monorepo_url')
export MONOREPO_AUTH
MONOREPO_AUTH=$(echo "$config_json" | jq -r '.josh.monorepo_auth // "https"')
export BOT_NAME
BOT_NAME=$(echo "$config_json" | jq -r '.bot.name')
export BOT_EMAIL
@@ -30,13 +40,35 @@ parse_config() {
[ -n "$JOSH_PROXY_URL" ] && [ "$JOSH_PROXY_URL" != "null" ] || die "josh.proxy_url missing in config"
[ -n "$MONOREPO_PATH" ] && [ "$MONOREPO_PATH" != "null" ] || die "josh.monorepo_path missing in config"
[ -n "$MONOREPO_URL" ] && [ "$MONOREPO_URL" != "null" ] || die "josh.monorepo_url is required (added in schema_version 2). Set it to your monorepo's git clone URL, e.g. 'git@code.example.io:org/repo.git'."
[ "$MONOREPO_AUTH" = "ssh" ] || [ "$MONOREPO_AUTH" = "https" ] || die "josh.monorepo_auth must be 'ssh' or 'https' (got '${MONOREPO_AUTH}')"
[ -n "$BOT_TRAILER" ] && [ "$BOT_TRAILER" != "null" ] || die "bot.trailer missing in config"
# Derive monorepo gitea_host and repo_path from monorepo_url (mirrors target derivation below)
local monorepo_meta
monorepo_meta=$(jq -nr --arg url "$MONOREPO_URL" '
$url as $u |
if ($u | test("^ssh://")) then
($u | capture("ssh://[^@]*@(?<h>[^/]+)/(?<p>.+?)(\\.git)?$") // {h: "", p: ""})
elif ($u | test("^git@")) then
($u | capture("git@(?<h>[^:/]+)[:/](?<p>.+?)(\\.git)?$") // {h: "", p: ""})
elif ($u | test("^https?://")) then
($u | capture("https?://(?<h>[^/]+)/(?<p>.+?)(\\.git)?$") // {h: "", p: ""})
else
{h: "", p: ""}
end | "\(.h) \(.p)"')
export MONOREPO_GITEA_HOST="${monorepo_meta% *}"
export MONOREPO_REPO_PATH="${monorepo_meta#* }"
[ -n "$MONOREPO_GITEA_HOST" ] || die "Could not derive gitea host from josh.monorepo_url='${MONOREPO_URL}' (expected git@host:org/repo.git or https://host/org/repo.git)"
# Enrich targets with derived fields (gitea_host, subrepo_repo_path, auto josh_filter)
export JOSH_SYNC_TARGETS
JOSH_SYNC_TARGETS=$(echo "$config_json" | jq '[.targets[] | . +
# Auto-derive josh_filter from subfolder if not set
(if (.josh_filter // "") == "" then
# When exclude patterns are present, append inline :exclude[::p1,::p2,...] to the filter
(if (.exclude // [] | length) > 0 then
{josh_filter: (":/" + .subfolder + ":exclude[" + (.exclude | map("::" + .) | join(",")) + "]")}
elif (.josh_filter // "") == "" then
{josh_filter: (":/" + .subfolder)}
else {} end) +
# Derive gitea_host and subrepo_repo_path from subrepo_url
@@ -65,10 +97,8 @@ parse_config() {
export GITEA_TOKEN="${GITEA_TOKEN:-${SYNC_BOT_TOKEN:-}}"
export BOT_USER="${BOT_USER:-${SYNC_BOT_USER:-}}"
# Monorepo API URL (derived from first target's host, overridable via env)
local gitea_host
gitea_host=$(echo "$JOSH_SYNC_TARGETS" | jq -r '.[0].gitea_host')
export MONOREPO_API="${MONOREPO_API:-https://${gitea_host}/api/v1/repos/${MONOREPO_PATH}}"
# Monorepo API URL (derived from monorepo_url's host, overridable via env for custom API endpoints)
export MONOREPO_API="${MONOREPO_API:-https://${MONOREPO_GITEA_HOST}/api/v1/repos/${MONOREPO_PATH}}"
log "INFO" "Config loaded: $(echo "$JOSH_SYNC_TARGETS" | jq 'length') target(s)"
}

View File

@@ -11,7 +11,7 @@
# ─── Forward Sync: Monorepo → Subrepo ──────────────────────────────
#
# Returns: fresh | skip | clean | lease-rejected | conflict
# Returns: fresh | skip | clean | lease-rejected | conflict | unrelated
forward_sync() {
local mono_branch="$SYNC_BRANCH_MONO"
@@ -97,7 +97,14 @@ ${BOT_TRAILER}: forward/${mono_branch}/$(date -u +%Y-%m-%dT%H:%M:%SZ)" >&2
fi
else
# Conflict!
# Check: unrelated histories (filter change) vs normal merge conflict
if ! git merge-base "subrepo/${subrepo_branch}" "$mono_head" >/dev/null 2>&1; then
log "INFO" "No common ancestor — histories are unrelated (filter change?)"
echo "unrelated"
return
fi
# Normal merge conflict
local conflicted
conflicted=$(git diff --name-only --diff-filter=U 2>/dev/null || echo "(unknown)")
git merge --abort
@@ -115,7 +122,14 @@ ${BOT_TRAILER}: forward/${mono_branch}/$(date -u +%Y-%m-%dT%H:%M:%SZ)" >&2
local pr_body conflicted_list
# shellcheck disable=SC2001
conflicted_list=$(echo "$conflicted" | sed 's/^/- /')
pr_body="## Sync Conflict\n\nMonorepo \`${mono_branch}\` has changes that conflict with \`${subrepo_branch}\`.\n\n**Conflicted files:**\n${conflicted_list}\n\nPlease resolve and merge this PR to complete the sync."
pr_body="## Sync Conflict
Monorepo \`${mono_branch}\` has changes that conflict with \`${subrepo_branch}\`.
**Conflicted files:**
${conflicted_list}
Please resolve and merge this PR to complete the sync."
create_pr "${SUBREPO_API}" "${SUBREPO_TOKEN}" \
"$subrepo_branch" "$conflict_branch" \
@@ -128,10 +142,93 @@ ${BOT_TRAILER}: forward/${mono_branch}/$(date -u +%Y-%m-%dT%H:%M:%SZ)" >&2
fi
}
# ─── Filter Change Reconciliation ─────────────────────────────────
# When the josh filter changes (e.g., exclude patterns added/removed),
# josh-proxy recomputes filtered history with new SHAs. This creates a
# merge commit on the subrepo that connects old and new histories,
# re-establishing shared ancestry without a destructive force-push.
# Returns: reconciled | lease-rejected
reconcile_filter_change() {
local mono_branch="$SYNC_BRANCH_MONO"
local subrepo_branch="$SYNC_BRANCH_SUBREPO"
local work_dir
work_dir=$(mktemp -d)
# shellcheck disable=SC2064 # Intentional early expansion — work_dir is local
trap "rm -rf '$work_dir'" EXIT
log "INFO" "=== Filter change reconciliation: ${mono_branch} ==="
# 1. Clone subrepo
git clone "$(subrepo_auth_url)" \
--branch "$subrepo_branch" --single-branch \
"${work_dir}/subrepo" || die "Failed to clone subrepo"
cd "${work_dir}/subrepo" || exit
git config user.name "$BOT_NAME"
git config user.email "$BOT_EMAIL"
local subrepo_head
subrepo_head=$(git rev-parse HEAD)
log "INFO" "Subrepo HEAD: ${subrepo_head:0:12}"
# 2. Fetch josh-proxy filtered view (new filter)
git remote add josh-filtered "$(josh_auth_url)"
git fetch josh-filtered "$mono_branch" || die "Failed to fetch from josh-proxy"
local josh_head josh_tree
josh_head=$(git rev-parse "josh-filtered/${mono_branch}")
# shellcheck disable=SC1083 # {tree} is git syntax, not shell brace expansion
josh_tree=$(git rev-parse "josh-filtered/${mono_branch}^{tree}")
log "INFO" "Josh-proxy HEAD (new filter): ${josh_head:0:12}"
# 3. Check if trees are already identical (filter change had no effect)
local subrepo_tree
# shellcheck disable=SC1083
subrepo_tree=$(git rev-parse "HEAD^{tree}")
if [ "$josh_tree" = "$subrepo_tree" ]; then
log "INFO" "Trees identical after filter change — no reconciliation needed"
echo "skip"
return
fi
# 4. Create merge commit: josh-proxy HEAD (first parent) + subrepo HEAD, with josh-proxy's tree
# Josh follows first-parent traversal — josh-filtered MUST be first so josh can map
# the history back to the monorepo. Old subrepo history hangs off parent 2.
local merge_commit
merge_commit=$(git commit-tree "$josh_tree" \
-p "$josh_head" \
-p "$subrepo_head" \
-m "Sync: filter configuration updated
${BOT_TRAILER}: filter-change/${mono_branch}/$(date -u +%Y-%m-%dT%H:%M:%SZ)")
git reset --hard "$merge_commit" >&2
log "INFO" "Created reconciliation merge: ${merge_commit:0:12}"
# 5. Record lease and push
local subrepo_sha
subrepo_sha=$(subrepo_ls_remote "$subrepo_branch")
if git push \
--force-with-lease="refs/heads/${subrepo_branch}:${subrepo_sha}" \
"$(subrepo_auth_url)" \
"HEAD:refs/heads/${subrepo_branch}"; then
log "INFO" "Filter change reconciled — shared ancestry re-established"
echo "reconciled"
else
log "WARN" "Force-with-lease rejected — subrepo changed during reconciliation"
echo "lease-rejected"
fi
}
# ─── Reverse Sync: Subrepo → Monorepo ──────────────────────────────
#
# Always creates a PR on the monorepo — never pushes directly.
# Returns: skip | pr-created | josh-rejected
# Returns: skip | skip-dirty | pr-created | josh-rejected
# skip — trees identical, nothing to do
# skip-dirty — trees differ but no human commits found (don't update state)
reverse_sync() {
local mono_branch="$SYNC_BRANCH_MONO"
@@ -143,6 +240,8 @@ reverse_sync() {
log "INFO" "=== Reverse sync: subrepo/${subrepo_branch} → mono/${mono_branch} ==="
local force_mode="${SYNC_REVERSE_FORCE:-0}"
# 1. Clone subrepo
git clone "$(subrepo_auth_url)" \
--branch "$subrepo_branch" --single-branch \
@@ -156,31 +255,190 @@ reverse_sync() {
git remote add mono-filtered "$(josh_auth_url)"
git fetch mono-filtered "$mono_branch" || die "Failed to fetch from josh-proxy"
# 3. Find new human commits (excludes bot commits from forward sync)
local human_commits
human_commits=$(git log "mono-filtered/${mono_branch}..HEAD" \
--oneline --invert-grep --grep="^${BOT_TRAILER}:" 2>/dev/null || echo "")
# 3. Compare trees — skip if subrepo matches josh-filtered view
local subrepo_tree josh_tree
# shellcheck disable=SC1083 # {tree} is git syntax, not shell brace expansion
subrepo_tree=$(git rev-parse "HEAD^{tree}")
# shellcheck disable=SC1083
josh_tree=$(git rev-parse "mono-filtered/${mono_branch}^{tree}")
if [ -z "$human_commits" ]; then
log "INFO" "No new human commits in subrepo — nothing to sync"
if [ "$subrepo_tree" = "$josh_tree" ]; then
log "INFO" "Subrepo tree matches josh-filtered view — nothing to sync"
echo "skip"
return
fi
# 4. Find new human commits (excludes bot commits from forward sync)
# Uses merge-base + --ancestry-path to restrict to the direct lineage.
# merge-base is needed because mono-filtered may have advanced past the
# last forward sync (any monorepo commit, even outside the subfolder,
# causes josh to create a new filtered commit). Using mono-filtered
# directly with --ancestry-path returns empty when it's not an ancestor
# of HEAD. merge-base always finds the last connected point.
local base human_commits
base=$(git merge-base "mono-filtered/${mono_branch}" HEAD 2>/dev/null || echo "")
if [ -n "$base" ]; then
human_commits=$(git log --ancestry-path "${base}..HEAD" \
--oneline --invert-grep --grep="^${BOT_TRAILER}:" 2>/dev/null || echo "")
else
# No common ancestor (first sync or fully diverged) — skip ancestry-path
human_commits=$(git log "mono-filtered/${mono_branch}..HEAD" \
--oneline --invert-grep --grep="^${BOT_TRAILER}:" 2>/dev/null || echo "")
fi
if [ -z "$human_commits" ]; then
if [ "$force_mode" = "1" ]; then
log "WARN" "No human commits found — force mode: continuing anyway"
else
log "INFO" "No new human commits in subrepo — nothing to sync"
echo "skip-dirty"
return
fi
fi
log "INFO" "New human commits to sync:"
echo "$human_commits" >&2
# 4. Push through josh to a staging branch
# 5. Push through josh to a staging branch
local ts
ts=$(date +%Y%m%d-%H%M%S)
local staging_branch="auto-sync/subrepo-${subrepo_branch}-${ts}"
local push_ref="HEAD"
local linearized=false
if git push -o "base=${mono_branch}" "$(josh_auth_url)" "HEAD:refs/heads/${staging_branch}"; then
log "INFO" "Pushed to staging branch via josh: ${staging_branch}"
if ! git push -o "base=${mono_branch}" "$(josh_auth_url)" "${push_ref}:refs/heads/${staging_branch}" 2>/dev/null; then
# Josh rejects pushes when the history contains merge commits whose
# parents it cannot map (e.g., merges of staging/feature branches that
# include auto-sync history). Fall back to linearizing: cherry-pick
# regular commits as-is, squash only the problematic merge commits.
# See ADR-011.
log "WARN" "Direct push failed — linearizing history (cherry-pick + squash merges)"
log "INFO" "─── linearize start ───"
# 5. Create PR on monorepo (NEVER direct push)
local original_head
original_head=$(git rev-parse HEAD)
git checkout -b "tmp-linearize-$$" "mono-filtered/${mono_branch}" >/dev/null 2>&1 \
|| die "Failed to create linearize branch"
# Reuse merge-base from step 4 (same reasoning: mono-filtered may have
# advanced past the last forward sync, breaking --ancestry-path).
local commit_shas
if [ -n "$base" ]; then
commit_shas=$(git log --ancestry-path "${base}..${original_head}" \
--reverse --format="%H" --invert-grep --grep="^${BOT_TRAILER}:" 2>/dev/null || echo "")
else
commit_shas=$(git log "mono-filtered/${mono_branch}..${original_head}" \
--reverse --format="%H" --invert-grep --grep="^${BOT_TRAILER}:" 2>/dev/null || echo "")
fi
local sha
for sha in $commit_shas; do
local parent_count
parent_count=$(git cat-file -p "$sha" | grep -c "^parent ")
local short_msg
short_msg=$(git log -1 --format="%s" "$sha")
if [ "$parent_count" -le 1 ]; then
# Regular commit — cherry-pick to preserve message and authorship
if git cherry-pick "$sha" >/dev/null 2>&1; then
log "INFO" " cherry-pick ${sha:0:7}: ${short_msg}"
else
log "WARN" " conflict ${sha:0:7} — applying patch from parent"
git cherry-pick --abort 2>/dev/null
local msg author_name author_email author_date
msg=$(git log -1 --format="%B" "$sha")
author_name=$(git log -1 --format="%an" "$sha")
author_email=$(git log -1 --format="%ae" "$sha")
author_date=$(git log -1 --format="%aI" "$sha")
# Diff from the commit's own parent — NOT from HEAD (which is
# the linearize branch and may have a different base).
git diff "${sha}^" "$sha" | git apply --index >/dev/null 2>&1 \
|| die "Failed to apply diff for ${sha:0:7}"
GIT_AUTHOR_NAME="$author_name" GIT_AUTHOR_EMAIL="$author_email" \
GIT_AUTHOR_DATE="$author_date" \
git commit -m "$msg" >/dev/null 2>&1 || die "Failed to commit ${sha:0:7}"
log "INFO" " applied ${sha:0:7}: ${short_msg}"
fi
else
# Merge commit — cherry-pick relative to first parent (squashes the merge)
local msg author_name author_email author_date
msg=$(git log -1 --format="%B" "$sha")
author_name=$(git log -1 --format="%an" "$sha")
author_email=$(git log -1 --format="%ae" "$sha")
author_date=$(git log -1 --format="%aI" "$sha")
if git cherry-pick -m 1 "$sha" >/dev/null 2>&1; then
log "INFO" " squash ${sha:0:7}: ${short_msg}"
else
# Cherry-pick failed — distinguish empty result (changes already
# applied by a prior cherry-pick) from a real conflict.
if git diff --cached --quiet 2>/dev/null && git diff --quiet 2>/dev/null; then
# Working tree clean — the merge changes are already present
git cherry-pick --abort 2>/dev/null || git reset --hard HEAD >/dev/null 2>&1
log "INFO" " skip ${sha:0:7}: ${short_msg} (already applied)"
else
# Real conflict — apply diff from first parent (what the merge
# actually introduced, NOT from HEAD which would capture all
# tree differences).
git cherry-pick --abort 2>/dev/null
git diff "${sha}^1" "$sha" | git apply --index >/dev/null 2>&1 \
|| die "Failed to apply merge diff for ${sha:0:7}"
GIT_AUTHOR_NAME="$author_name" GIT_AUTHOR_EMAIL="$author_email" \
GIT_AUTHOR_DATE="$author_date" \
git commit -m "$msg" >/dev/null 2>&1 || die "Failed to commit merge ${sha:0:7}"
log "INFO" " squash ${sha:0:7}: ${short_msg}"
fi
fi
fi
done
log "INFO" "─── linearize done ────"
push_ref="HEAD"
linearized=true
git push -o "base=${mono_branch}" "$(josh_auth_url)" \
"${push_ref}:refs/heads/${staging_branch}" \
|| { log "ERROR" "Josh rejected linearized push — check josh-proxy logs"; echo "josh-rejected"; return; }
fi
log "INFO" "Pushed to staging branch via josh: ${staging_branch}${linearized:+ (linearized)}"
# 6. Publish: force-push directly (--force) or create PR (default)
if [ "$force_mode" = "1" ]; then
log "WARN" "Force mode: pushing ${staging_branch} directly to mono/${mono_branch} — bypassing PR"
local mono_dir="${work_dir}/monorepo"
git clone "$(mono_auth_url)" \
--branch "$staging_branch" --single-branch \
"$mono_dir" || die "Failed to clone monorepo staging branch for force-push"
git -C "$mono_dir" push --force "$(mono_auth_url)" \
"HEAD:refs/heads/${mono_branch}" \
|| die "Failed to force-push to mono/${mono_branch}"
log "INFO" "Force-pushed mono/${staging_branch} → mono/${mono_branch}"
echo "forced"
else
local pr_body
pr_body="## Subrepo changes\n\nNew commits from subrepo \`${subrepo_branch}\`:\n\n\`\`\`\n${human_commits}\n\`\`\`\n\n**Review checklist:**\n- [ ] Changes scoped to synced subfolder\n- [ ] No leaked credentials or environment-specific config\n- [ ] CI passes"
local linearize_note=""
if [ "$linearized" = true ]; then
linearize_note="
> **Note:** Merge commits were squashed during sync because the subrepo
> history contained merges that josh-proxy cannot map (see ADR-011).
> Regular commits are preserved individually."
fi
pr_body="## Subrepo changes
New commits from subrepo \`${subrepo_branch}\`:
\`\`\`
${human_commits}
\`\`\`
${linearize_note}
**Review checklist:**
- [ ] Changes scoped to synced subfolder
- [ ] No leaked credentials or environment-specific config
- [ ] CI passes"
create_pr "${MONOREPO_API}" "${GITEA_TOKEN}" \
"$mono_branch" "$staging_branch" \
@@ -190,9 +448,6 @@ reverse_sync() {
log "INFO" "Reverse sync PR created on monorepo"
echo "pr-created"
else
log "ERROR" "Josh rejected push — check josh-proxy logs"
echo "josh-rejected"
fi
}
@@ -220,12 +475,7 @@ initial_import() {
log "INFO" "=== Initial import: subrepo/${subrepo_branch} → mono/${mono_branch}:${subfolder}/ ==="
# 1. Clone monorepo (directly, not through josh — we need the real repo)
local mono_auth_url
local api_host_path
api_host_path=$(echo "$MONOREPO_API" | sed 's|https://||; s|/api/v1/repos/|/|')
mono_auth_url="https://${BOT_USER}:${GITEA_TOKEN}@${api_host_path}.git"
git clone "$mono_auth_url" \
git clone "$(mono_auth_url)" \
--branch "$mono_branch" --single-branch \
"${work_dir}/monorepo" || die "Failed to clone monorepo"
@@ -268,7 +518,14 @@ ${BOT_TRAILER}: import/${JOSH_SYNC_TARGET_NAME}/${ts}" >&2
# 5. Create PR on monorepo
local pr_body
pr_body="## Initial import\n\nImporting existing subrepo \`${subrepo_branch}\` (${file_count} files) into \`${subfolder}/\`.\n\n**Review checklist:**\n- [ ] Content looks correct\n- [ ] No leaked credentials or environment-specific config\n- [ ] CI passes"
pr_body="## Initial import
Importing existing subrepo \`${subrepo_branch}\` (${file_count} files) into \`${subfolder}/\`.
**Review checklist:**
- [ ] Content looks correct
- [ ] No leaked credentials or environment-specific config
- [ ] CI passes"
create_pr "${MONOREPO_API}" "${GITEA_TOKEN}" \
"$mono_branch" "$staging_branch" \

View File

@@ -3,11 +3,16 @@
"title": "josh-sync configuration",
"description": "Configuration for bidirectional monorepo ↔ subrepo sync via josh-proxy",
"type": "object",
"required": ["josh", "targets", "bot"],
"required": ["schema_version", "josh", "targets", "bot"],
"properties": {
"schema_version": {
"type": "integer",
"const": 2,
"description": "Schema version. Required. Must be 2 for josh-sync v2.x configs."
},
"josh": {
"type": "object",
"required": ["proxy_url", "monorepo_path"],
"required": ["proxy_url", "monorepo_path", "monorepo_url"],
"properties": {
"proxy_url": {
"type": "string",
@@ -16,7 +21,17 @@
},
"monorepo_path": {
"type": "string",
"description": "Repo path as josh sees it (org/repo)"
"description": "Repo path as josh-proxy sees it (org/repo)"
},
"monorepo_url": {
"type": "string",
"description": "Monorepo git clone URL — git@host:org/repo.git (SSH) or https://host/org/repo.git. The gitea host is derived from this URL."
},
"monorepo_auth": {
"type": "string",
"enum": ["https", "ssh"],
"default": "https",
"description": "Auth method for cloning the monorepo. SSH uses the user's ssh-agent / ~/.ssh/config; HTTPS injects ${SYNC_BOT_USER}:${SYNC_BOT_TOKEN} into the URL."
}
}
},
@@ -70,6 +85,12 @@
"items": { "type": "string" },
"default": [],
"description": "Branches that only sync mono → subrepo (never reverse)"
},
"exclude": {
"type": "array",
"items": { "type": "string" },
"default": [],
"description": "File/directory patterns to exclude from sync via josh :exclude filter. Josh pattern syntax: 'dir/' for directories, '*.ext' for globs, '**/dir/' for nested matches. Patterns are embedded inline in the josh-proxy URL."
}
}
}

View File

@@ -1,7 +1,9 @@
# Test fixture: minimal single-target config
schema_version: 2
josh:
proxy_url: "https://josh.test.local"
monorepo_path: "org/repo"
monorepo_url: "git@gitea.test.local:org/repo.git"
targets:
- name: "example"

View File

@@ -1,7 +1,9 @@
# Test fixture: multi-target config
schema_version: 2
josh:
proxy_url: "https://josh.test.local"
monorepo_path: "org/monorepo"
monorepo_url: "git@gitea.test.local:org/monorepo.git"
targets:
- name: "app-a"

View File

@@ -37,3 +37,39 @@ setup() {
url=$(subrepo_auth_url)
[ "$url" = "git@gitea.test.local:ext/app-a.git" ]
}
@test "mono_auth_url returns bare URL for SSH" {
export MONOREPO_AUTH="ssh"
export MONOREPO_URL="git@code.example.io:org/monorepo.git"
local url
url=$(mono_auth_url)
[ "$url" = "git@code.example.io:org/monorepo.git" ]
}
@test "mono_auth_url injects credentials for HTTPS git@-form monorepo_url" {
export MONOREPO_AUTH="https"
export MONOREPO_URL="git@code.example.io:org/monorepo.git"
local url
url=$(mono_auth_url)
[ "$url" = "https://sync-bot:test-token-123@code.example.io/org/monorepo.git" ]
}
@test "mono_auth_url injects credentials for HTTPS https-form monorepo_url" {
export MONOREPO_AUTH="https"
export MONOREPO_URL="https://code.example.io/org/monorepo.git"
local url
url=$(mono_auth_url)
[ "$url" = "https://sync-bot:test-token-123@code.example.io/org/monorepo.git" ]
}
@test "mono_auth_url defaults to HTTPS when MONOREPO_AUTH unset" {
unset MONOREPO_AUTH
export MONOREPO_URL="https://code.example.io/org/monorepo.git"
local url
url=$(mono_auth_url)
[ "$url" = "https://sync-bot:test-token-123@code.example.io/org/monorepo.git" ]
}

View File

@@ -104,3 +104,151 @@ setup() {
run bash -c 'source "$JOSH_SYNC_ROOT/lib/core.sh"; source "$JOSH_SYNC_ROOT/lib/config.sh"; parse_config "nonexistent.yml"'
[ "$status" -ne 0 ]
}
@test "parse_config fails when schema_version is missing (no v1 fallback)" {
cd "$(mktemp -d)"
cat > .josh-sync.yml <<'YAML'
josh:
proxy_url: "https://josh.test.local"
monorepo_path: "org/repo"
monorepo_url: "git@gitea.test.local:org/repo.git"
targets:
- name: "x"
subfolder: "x"
subrepo_url: "git@gitea.test.local:ext/x.git"
branches: { main: main }
bot:
name: "b"
email: "b@b"
trailer: "T"
YAML
run bash -c 'source "$JOSH_SYNC_ROOT/lib/core.sh"; source "$JOSH_SYNC_ROOT/lib/config.sh"; parse_config ".josh-sync.yml"'
[ "$status" -ne 0 ]
[[ "$output" == *"schema_version missing"* ]]
}
@test "parse_config rejects schema_version: 1" {
cd "$(mktemp -d)"
cat > .josh-sync.yml <<'YAML'
schema_version: 1
josh:
proxy_url: "https://josh.test.local"
monorepo_path: "org/repo"
monorepo_url: "git@gitea.test.local:org/repo.git"
targets:
- name: "x"
subfolder: "x"
subrepo_url: "git@gitea.test.local:ext/x.git"
branches: { main: main }
bot:
name: "b"
email: "b@b"
trailer: "T"
YAML
run bash -c 'source "$JOSH_SYNC_ROOT/lib/core.sh"; source "$JOSH_SYNC_ROOT/lib/config.sh"; parse_config ".josh-sync.yml"'
[ "$status" -ne 0 ]
[[ "$output" == *"Unsupported schema_version"* ]] || [[ "$output" == *"requires schema_version 2"* ]]
}
@test "parse_config fails when josh.monorepo_url is missing" {
cd "$(mktemp -d)"
cat > .josh-sync.yml <<'YAML'
schema_version: 2
josh:
proxy_url: "https://josh.test.local"
monorepo_path: "org/repo"
targets:
- name: "x"
subfolder: "x"
subrepo_url: "git@gitea.test.local:ext/x.git"
branches: { main: main }
bot:
name: "b"
email: "b@b"
trailer: "T"
YAML
run bash -c 'source "$JOSH_SYNC_ROOT/lib/core.sh"; source "$JOSH_SYNC_ROOT/lib/config.sh"; parse_config ".josh-sync.yml"'
[ "$status" -ne 0 ]
[[ "$output" == *"monorepo_url"* ]]
}
@test "parse_config exports MONOREPO_URL, MONOREPO_AUTH, MONOREPO_GITEA_HOST, MONOREPO_REPO_PATH from SSH form" {
cd "$(mktemp -d)"
cp "$FIXTURES/minimal.yml" .josh-sync.yml
parse_config ".josh-sync.yml"
[ "$MONOREPO_URL" = "git@gitea.test.local:org/repo.git" ]
[ "$MONOREPO_AUTH" = "https" ]
[ "$MONOREPO_GITEA_HOST" = "gitea.test.local" ]
[ "$MONOREPO_REPO_PATH" = "org/repo" ]
}
@test "parse_config derives MONOREPO_GITEA_HOST from HTTPS monorepo_url" {
cd "$(mktemp -d)"
cat > .josh-sync.yml <<'YAML'
schema_version: 2
josh:
proxy_url: "https://josh.test.local"
monorepo_path: "org/repo"
monorepo_url: "https://gitea.example.io/org/repo.git"
targets:
- name: "x"
subfolder: "x"
subrepo_url: "git@gitea.test.local:ext/x.git"
branches: { main: main }
bot:
name: "b"
email: "b@b"
trailer: "T"
YAML
parse_config ".josh-sync.yml"
[ "$MONOREPO_GITEA_HOST" = "gitea.example.io" ]
[ "$MONOREPO_REPO_PATH" = "org/repo" ]
}
@test "parse_config derives MONOREPO_API from monorepo_url's host (cross-host: monorepo and target on different hosts)" {
cd "$(mktemp -d)"
cat > .josh-sync.yml <<'YAML'
schema_version: 2
josh:
proxy_url: "https://josh.example.io"
monorepo_path: "org/repo"
monorepo_url: "git@code.example.io:org/repo.git"
targets:
- name: "x"
subfolder: "x"
subrepo_url: "git@other-host.io:ext/x.git"
branches: { main: main }
bot:
name: "b"
email: "b@b"
trailer: "T"
YAML
parse_config ".josh-sync.yml"
[ "$MONOREPO_API" = "https://code.example.io/api/v1/repos/org/repo" ]
}
@test "parse_config rejects invalid monorepo_auth value" {
cd "$(mktemp -d)"
cat > .josh-sync.yml <<'YAML'
schema_version: 2
josh:
proxy_url: "https://josh.test.local"
monorepo_path: "org/repo"
monorepo_url: "git@gitea.test.local:org/repo.git"
monorepo_auth: "weird"
targets:
- name: "x"
subfolder: "x"
subrepo_url: "git@gitea.test.local:ext/x.git"
branches: { main: main }
bot:
name: "b"
email: "b@b"
trailer: "T"
YAML
run bash -c 'source "$JOSH_SYNC_ROOT/lib/core.sh"; source "$JOSH_SYNC_ROOT/lib/config.sh"; parse_config ".josh-sync.yml"'
[ "$status" -ne 0 ]
[[ "$output" == *"monorepo_auth"* ]]
}