Compare commits
9 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 5e9b134168 | |||
| 1cb23a4411 | |||
| 479b592076 | |||
| 65cc8eaf8a | |||
| 4861179c6f | |||
| c9b37f2477 | |||
| d3799187a1 | |||
| b4eaa47ef6 | |||
| d6f334b861 |
62
CHANGELOG.md
62
CHANGELOG.md
@@ -1,7 +1,65 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## 1.5.0
|
||||||
|
|
||||||
|
### Breaking Changes
|
||||||
|
|
||||||
|
_None._
|
||||||
|
|
||||||
|
### Features
|
||||||
|
|
||||||
|
- **`--force` flag for reverse sync**: `josh-sync sync --reverse --force` bypasses
|
||||||
|
the PR step and force-pushes the subrepo's state directly onto the monorepo branch.
|
||||||
|
Also skips the `skip-dirty` guard. Only valid with `--reverse`. Returns status `forced`.
|
||||||
|
|
||||||
|
## 1.4.0
|
||||||
|
|
||||||
|
### Breaking Changes
|
||||||
|
|
||||||
|
_None._
|
||||||
|
|
||||||
|
### Features
|
||||||
|
|
||||||
|
- **`schema_version` config field**: New optional integer field in `.josh-sync.yml`. Defaults to `1` when absent. josh-sync fails fast with a clear error if an unsupported version is present — safe path for future config migrations without silent misinterpretation.
|
||||||
|
|
||||||
|
### Fixes
|
||||||
|
|
||||||
|
- **Linearize fallback applying wrong diff**: Fallback for failed `cherry-pick` was diffing from `HEAD` to the commit's full tree instead of from the commit's own parent, causing unrelated file deletions. Now diffs from the commit's own parent (`${sha}^` / `${sha}^1`).
|
||||||
|
- **Linearize fallback for already-applied merge commits**: When `cherry-pick -m 1` produces an empty result because a merge's changes were already applied by a prior cherry-pick, the commit is now skipped instead of attempting a failing `git apply`.
|
||||||
|
- **Linearize fallback output**: Suppressed git noise from cherry-pick, checkout, and commit during linearization; added delimiters and consistent indented log lines.
|
||||||
|
|
||||||
|
### Docs
|
||||||
|
|
||||||
|
- Workflow guide: removed redundant step-by-step subsections (table already covers them); tightened ADR-011 alternatives; removed stale version pins.
|
||||||
|
- README: added Versioning section with semver policy table and floating tag explanation.
|
||||||
|
|
||||||
|
## 1.3.0
|
||||||
|
|
||||||
|
### Breaking Changes
|
||||||
|
|
||||||
|
_None._
|
||||||
|
|
||||||
|
### Features
|
||||||
|
|
||||||
|
- **Linearize fallback for reverse sync**: When josh-proxy rejects a push due to unmappable merge commits, reverse sync automatically falls back to linearizing the history — cherry-picking regular commits individually and squashing only the problematic merge commits via `cherry-pick -m 1`. Preserves individual commit granularity while handling complex merge topologies that josh cannot map. PR body notes when linearization was used.
|
||||||
|
|
||||||
|
### Fixes
|
||||||
|
|
||||||
|
- **Reverse sync human commit detection**: Fixed `--ancestry-path` returning empty when `mono-filtered` has advanced past the last forward sync. Any monorepo commit (even outside the subfolder) causes josh to create a new filtered commit, breaking the ancestry path to subrepo HEAD. Now uses `git merge-base` to find the last connected point.
|
||||||
|
- **State not updated on false skip**: When trees differ but no human commits are found, reverse sync now returns `skip-dirty` instead of `skip`. State is not updated, so the system retries on the next run instead of silently dropping commits.
|
||||||
|
|
||||||
|
### Docs
|
||||||
|
|
||||||
|
- Expanded troubleshooting guide for "Josh rejected push" with root cause analysis and prevention advice.
|
||||||
|
- Added ADR-011: Linearize fallback for reverse sync.
|
||||||
|
- Added recommended Git workflow section to the guide.
|
||||||
|
|
||||||
## 1.2.0
|
## 1.2.0
|
||||||
|
|
||||||
|
### Breaking Changes
|
||||||
|
|
||||||
|
_None._
|
||||||
|
|
||||||
### Features
|
### Features
|
||||||
|
|
||||||
- **File exclusion**: `exclude` config field removes files/directories from the subrepo at the josh-proxy transport layer. Patterns are embedded inline in the josh-proxy URL using `:exclude[::pattern,...]` syntax — no extra files to generate or commit.
|
- **File exclusion**: `exclude` config field removes files/directories from the subrepo at the josh-proxy transport layer. Patterns are embedded inline in the josh-proxy URL using `:exclude[::pattern,...]` syntax — no extra files to generate or commit.
|
||||||
@@ -21,6 +79,10 @@
|
|||||||
|
|
||||||
## 1.1.0
|
## 1.1.0
|
||||||
|
|
||||||
|
### Breaking Changes
|
||||||
|
|
||||||
|
_None._
|
||||||
|
|
||||||
### Features
|
### Features
|
||||||
|
|
||||||
- **`onboard` command**: Interactive, resumable workflow for importing existing subrepos into the monorepo. Walks through: prerequisites check, import (creates PRs), wait for merge, reset (pushes josh-filtered history). Checkpoint/resume at every step.
|
- **`onboard` command**: Interactive, resumable workflow for importing existing subrepos into the monorepo. Walks through: prerequisites check, import (creates PRs), wait for merge, reset (pushes josh-filtered history). Checkpoint/resume at every step.
|
||||||
|
|||||||
16
README.md
16
README.md
@@ -63,10 +63,22 @@ Run `josh-sync preflight` to validate your setup.
|
|||||||
- **[Architecture Decision Records](docs/adr/)** — Design rationale and trade-offs
|
- **[Architecture Decision Records](docs/adr/)** — Design rationale and trade-offs
|
||||||
- **[Changelog](CHANGELOG.md)** — Version history
|
- **[Changelog](CHANGELOG.md)** — Version history
|
||||||
|
|
||||||
|
## Versioning
|
||||||
|
|
||||||
|
josh-sync follows [semver](https://semver.org/):
|
||||||
|
|
||||||
|
| Bump | Meaning | Action required |
|
||||||
|
|------|---------|-----------------|
|
||||||
|
| **Patch** (1.x.**y**) | Bug fixes only | Safe to upgrade |
|
||||||
|
| **Minor** (1.**y**.0) | New optional config fields, new commands, new action inputs | Safe to upgrade; new features are opt-in |
|
||||||
|
| **Major** (**y**.0.0) | Removed or renamed config fields, CLI flags, action inputs, env vars, or state format | Read the **Breaking Changes** section in the changelog before upgrading |
|
||||||
|
|
||||||
|
CI workflows pin to a floating major tag (e.g. `@v1`). A breaking change bumps the major version and moves the tag — `@v1` workflows keep working until you explicitly update to `@v2`.
|
||||||
|
|
||||||
## CLI
|
## CLI
|
||||||
|
|
||||||
```
|
```
|
||||||
josh-sync sync [--forward|--reverse] [--target NAME[,NAME]] [--branch BRANCH]
|
josh-sync sync [--forward|--reverse] [--force] [--target NAME[,NAME]] [--branch BRANCH]
|
||||||
josh-sync preflight
|
josh-sync preflight
|
||||||
josh-sync import <target>
|
josh-sync import <target>
|
||||||
josh-sync reset <target>
|
josh-sync reset <target>
|
||||||
@@ -80,7 +92,7 @@ josh-sync state reset <target> [branch]
|
|||||||
## How It Works
|
## How It Works
|
||||||
|
|
||||||
- **Forward sync** (mono → subrepo): pushes directly if clean, creates conflict PR if not. Uses `--force-with-lease` for safety.
|
- **Forward sync** (mono → subrepo): pushes directly if clean, creates conflict PR if not. Uses `--force-with-lease` for safety.
|
||||||
- **Reverse sync** (subrepo → mono): always creates a PR, never pushes directly.
|
- **Reverse sync** (subrepo → mono): creates a PR by default. Add `--force` to bypass the PR and push directly to the mono branch (destructive).
|
||||||
- **File exclusion**: `exclude` patterns are embedded inline in the josh-proxy URL. Excluded files exist only in the monorepo.
|
- **File exclusion**: `exclude` patterns are embedded inline in the josh-proxy URL. Excluded files exist only in the monorepo.
|
||||||
- **Filter reconciliation**: Changing the exclude list auto-creates a merge commit that connects old and new histories — no force-push needed.
|
- **Filter reconciliation**: Changing the exclude list auto-creates a merge commit that connects old and new histories — no force-push needed.
|
||||||
- **Loop prevention**: `Josh-Sync-Origin:` git trailer filters out bot commits.
|
- **Loop prevention**: `Josh-Sync-Origin:` git trailer filters out bot commits.
|
||||||
|
|||||||
@@ -88,6 +88,7 @@ Global flags:
|
|||||||
Sync flags:
|
Sync flags:
|
||||||
--forward Forward only (mono → subrepo)
|
--forward Forward only (mono → subrepo)
|
||||||
--reverse Reverse only (subrepo → mono)
|
--reverse Reverse only (subrepo → mono)
|
||||||
|
--force Force-push subrepo state to mono branch (--reverse only, skips PR)
|
||||||
--target NAME Filter to target(s) — comma-separated for multiple (env: JOSH_SYNC_TARGET)
|
--target NAME Filter to target(s) — comma-separated for multiple (env: JOSH_SYNC_TARGET)
|
||||||
--branch BRANCH Filter to one branch
|
--branch BRANCH Filter to one branch
|
||||||
|
|
||||||
@@ -108,6 +109,7 @@ cmd_sync() {
|
|||||||
local filter_target="${JOSH_SYNC_TARGET:-}"
|
local filter_target="${JOSH_SYNC_TARGET:-}"
|
||||||
local filter_branch=""
|
local filter_branch=""
|
||||||
local config_file=".josh-sync.yml"
|
local config_file=".josh-sync.yml"
|
||||||
|
local force_flag=false
|
||||||
|
|
||||||
while [ $# -gt 0 ]; do
|
while [ $# -gt 0 ]; do
|
||||||
case "$1" in
|
case "$1" in
|
||||||
@@ -117,10 +119,18 @@ cmd_sync() {
|
|||||||
--branch) filter_branch="$2"; shift 2 ;;
|
--branch) filter_branch="$2"; shift 2 ;;
|
||||||
--config) config_file="$2"; shift 2 ;;
|
--config) config_file="$2"; shift 2 ;;
|
||||||
--debug) export JOSH_SYNC_DEBUG=1; shift ;;
|
--debug) export JOSH_SYNC_DEBUG=1; shift ;;
|
||||||
|
--force) force_flag=true; shift ;;
|
||||||
*) die "Unknown flag: $1" ;;
|
*) die "Unknown flag: $1" ;;
|
||||||
esac
|
esac
|
||||||
done
|
done
|
||||||
|
|
||||||
|
if [ "$force_flag" = true ] && [ "$direction" != "reverse" ]; then
|
||||||
|
die "--force requires --reverse"
|
||||||
|
fi
|
||||||
|
if [ "$force_flag" = true ]; then
|
||||||
|
export SYNC_REVERSE_FORCE=1
|
||||||
|
fi
|
||||||
|
|
||||||
parse_config "$config_file"
|
parse_config "$config_file"
|
||||||
|
|
||||||
# Forward sync
|
# Forward sync
|
||||||
@@ -259,6 +269,13 @@ _sync_direction() {
|
|||||||
echo "::error::Target ${target_name}, branch ${branch}: josh rejected push — check proxy logs"
|
echo "::error::Target ${target_name}, branch ${branch}: josh rejected push — check proxy logs"
|
||||||
continue
|
continue
|
||||||
fi
|
fi
|
||||||
|
if [ "$result" = "skip-dirty" ]; then
|
||||||
|
log "WARN" "Trees differ but no human commits — skipping state update (will retry)"
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
if [ "$result" = "forced" ]; then
|
||||||
|
log "WARN" "Target ${target_name}, branch ${branch}: force-pushed subrepo directly to mono — DESTRUCTIVE"
|
||||||
|
fi
|
||||||
|
|
||||||
# Update state (only on success)
|
# Update state (only on success)
|
||||||
local new_state
|
local new_state
|
||||||
|
|||||||
58
docs/adr/011-linearize-fallback-reverse.md
Normal file
58
docs/adr/011-linearize-fallback-reverse.md
Normal file
@@ -0,0 +1,58 @@
|
|||||||
|
# ADR-011: Linearize Fallback for Reverse Sync
|
||||||
|
|
||||||
|
**Status:** Accepted
|
||||||
|
**Date:** 2026-03
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
Josh-proxy rejects reverse sync pushes when the subrepo history contains merge commits whose parents it cannot map through the filter. This happens when:
|
||||||
|
|
||||||
|
1. A long-lived branch (e.g., `stage`) is merged into `main` via a merge commit
|
||||||
|
2. That branch contains auto-sync merge commits or criss-cross merges with `main`
|
||||||
|
3. Josh encounters the merge commit, tries to map both parents through the filter, and fails with a 500 error
|
||||||
|
|
||||||
|
This is a legitimate subrepo workflow — teams merge staging branches into main regularly. The sync tool should handle it without requiring teams to change their Git workflow.
|
||||||
|
|
||||||
|
### Alternatives considered
|
||||||
|
|
||||||
|
1. **Squash all commits into one**: Simple but destroys all commit granularity.
|
||||||
|
2. **Rewrite subrepo history**: Breaks josh's SHA mapping, forces all developers to re-clone.
|
||||||
|
3. **Linearize: cherry-pick regular commits, squash only merges**: Preserves individual commit granularity; only merge commits lose their multi-parent structure.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
When the direct push through josh-proxy fails, fall back to option 4: linearize the history by cherry-picking onto the josh-filtered base.
|
||||||
|
|
||||||
|
### How it works
|
||||||
|
|
||||||
|
1. Direct `git push` through josh-proxy is attempted first (existing behavior)
|
||||||
|
2. If it fails, create a temporary branch from `mono-filtered/<branch>`
|
||||||
|
3. Walk human commits (oldest-first, bot commits excluded) from the ancestry path:
|
||||||
|
- **Regular commits** (≤1 parent): `git cherry-pick <sha>` — preserves author, date, and message
|
||||||
|
- **Merge commits** (>1 parent): `git cherry-pick -m 1 <sha>` — applies the merge's diff relative to its first parent as a single commit
|
||||||
|
4. If cherry-pick conflicts (rare — usually due to ordering issues), fall back to `git diff | git apply` with the original author metadata
|
||||||
|
5. Push the linearized branch through josh-proxy
|
||||||
|
6. PR body includes a note explaining that merge commits were squashed
|
||||||
|
|
||||||
|
### What is preserved
|
||||||
|
|
||||||
|
- Individual non-merge commits (author, date, message, diff)
|
||||||
|
- The net effect of each merge commit (as a squashed single commit)
|
||||||
|
- The original commit list in the PR body for reference
|
||||||
|
|
||||||
|
### What is lost
|
||||||
|
|
||||||
|
- The multi-parent structure of merge commits (they become single-parent)
|
||||||
|
- The distinction between "changes introduced by the merge" vs "changes from each parent" — the merge is represented as its diff from first parent
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
**Positive:**
|
||||||
|
- Reverse sync no longer gets stuck on merge commits — handles the common case automatically
|
||||||
|
- Non-merge commits retain full granularity (no unnecessary squashing)
|
||||||
|
- Subrepo history is untouched — no rewriting, no broken sync relationship
|
||||||
|
- The PR body documents when linearization was used, so reviewers know
|
||||||
|
|
||||||
|
**Negative:**
|
||||||
|
- Merge commit semantics are lost on the monorepo side (they appear as regular commits)
|
||||||
|
- Adds complexity to the reverse sync path (two code paths: direct push and linearize fallback)
|
||||||
@@ -16,3 +16,4 @@ This directory contains Architecture Decision Records (ADRs) for josh-sync. Each
|
|||||||
| [008](008-first-parent-ordering.md) | First-parent ordering in reconciliation merges | Accepted |
|
| [008](008-first-parent-ordering.md) | First-parent ordering in reconciliation merges | Accepted |
|
||||||
| [009](009-tree-comparison-guard.md) | Tree comparison as sync skip guard | Accepted |
|
| [009](009-tree-comparison-guard.md) | Tree comparison as sync skip guard | Accepted |
|
||||||
| [010](010-onboard-checkpoint-resume.md) | Onboard workflow with checkpoint/resume | Accepted |
|
| [010](010-onboard-checkpoint-resume.md) | Onboard workflow with checkpoint/resume | Accepted |
|
||||||
|
| [011](011-linearize-fallback-reverse.md) | Linearize fallback for reverse sync | Accepted |
|
||||||
|
|||||||
@@ -518,6 +518,38 @@ Bot commits include a git trailer like `Josh-Sync-Origin: forward/main/2024-02-1
|
|||||||
|
|
||||||
Sync state is stored as JSON files on an orphan branch (`josh-sync-state`), one file per target/branch. This tracks the last-synced commit SHAs and timestamps to avoid re-syncing the same changes.
|
Sync state is stored as JSON files on an orphan branch (`josh-sync-state`), one file per target/branch. This tracks the last-synced commit SHAs and timestamps to avoid re-syncing the same changes.
|
||||||
|
|
||||||
|
## Recommended Git Workflow
|
||||||
|
|
||||||
|
Josh-proxy maps commits through a filter per-branch. It handles linear history and simple merges (short-lived feature branches) without issues. However, it **cannot map merge commits whose parents were created on the subrepo side** — because those commits were never pushed through josh and have no monorepo-side mapping.
|
||||||
|
|
||||||
|
This means cross-branch merges (e.g., `stage` → `main`) must happen on the monorepo side, where josh can filter the result cleanly.
|
||||||
|
|
||||||
|
### The rule
|
||||||
|
|
||||||
|
**The monorepo owns branch topology. The subrepo owns feature development.**
|
||||||
|
|
||||||
|
### What to do where
|
||||||
|
|
||||||
|
| Action | Where to do it | Why |
|
||||||
|
|--------|---------------|-----|
|
||||||
|
| Feature branch → `main` | **Subrepo** (PR, any merge strategy) | Short-lived branch with clean lineage — josh handles it |
|
||||||
|
| `stage` → `main` (promotion) | **Monorepo** | Cross-branch merge — forward sync propagates the result to both subrepo branches |
|
||||||
|
| `main` → `stage` (catch-up) | **Monorepo** | Same reason — avoids criss-cross merge history on subrepo |
|
||||||
|
| Hotfix to `main` | **Either side** | Single commit or small PR — works everywhere |
|
||||||
|
| Config/CI changes (monorepo-only) | **Monorepo** | Not synced to subrepo (use `exclude` for monorepo-only files) |
|
||||||
|
|
||||||
|
### What to avoid
|
||||||
|
|
||||||
|
- **Don't merge `stage` into `main` on the subrepo with a merge commit.** The merge parents include commits created on the subrepo side (forward sync merges, criss-cross merges) that josh has no mapping for. Josh rejects the push with a 500 error.
|
||||||
|
- **Don't merge `main` into `stage` on the subrepo.** Creates criss-cross merge history that causes the same josh mapping failure when `stage` is later merged back.
|
||||||
|
- **Don't rebase synced branches on the subrepo.** This rewrites commit SHAs that josh has already mapped, breaking the sync relationship.
|
||||||
|
|
||||||
|
### If you must merge cross-branch on the subrepo
|
||||||
|
|
||||||
|
Use a **squash merge**. A squash merge produces a single commit with one parent — josh can always map it. You lose the individual commit history on the target branch, but the sync goes through cleanly.
|
||||||
|
|
||||||
|
As a safety net, josh-sync automatically falls back to linearizing the history when josh rejects a push — cherry-picking regular commits individually and squashing only the problematic merge commits. See the [troubleshooting section](#josh-rejected-push-reverse-sync) and [ADR-011](adr/011-linearize-fallback-reverse.md).
|
||||||
|
|
||||||
## Excluding Files from Sync
|
## Excluding Files from Sync
|
||||||
|
|
||||||
Some files in the monorepo subfolder may not belong in the subrepo (e.g., monorepo-specific CI configs, internal tooling). The `exclude` config field removes these at the josh-proxy layer — excluded files never appear in the subrepo.
|
Some files in the monorepo subfolder may not belong in the subrepo (e.g., monorepo-specific CI configs, internal tooling). The `exclude` config field removes these at the josh-proxy layer — excluded files never appear in the subrepo.
|
||||||
@@ -616,7 +648,35 @@ Normal: the subrepo changed while sync was running. The next sync run will pick
|
|||||||
|
|
||||||
### "Josh rejected push" (reverse sync)
|
### "Josh rejected push" (reverse sync)
|
||||||
|
|
||||||
Josh-proxy couldn't map the push back to the monorepo. Check josh-proxy logs, verify the josh filter is correct. May indicate a history divergence — consider running `josh-sync reset <target>`.
|
Josh-proxy couldn't map the push back to the monorepo. This has two common causes:
|
||||||
|
|
||||||
|
#### Merge commits with unmappable parents
|
||||||
|
|
||||||
|
**Symptom:** Josh returns `500 Internal Server Error` with a message like:
|
||||||
|
|
||||||
|
```
|
||||||
|
rejecting merge with 2 parents:
|
||||||
|
"Merge pull request 'stage' (#30) from stage into main" (c4fa3c9...)
|
||||||
|
1) "Merge branch 'auto-sync/import-...'" (4bf8704...)
|
||||||
|
2) "Merge branch 'main' into stage" (d021654...)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Cause:** The subrepo has a merge commit whose parents josh-proxy cannot trace through its filter. This typically happens when:
|
||||||
|
- A long-lived branch (e.g., `stage`) is merged into `main` via a merge commit (not squash)
|
||||||
|
- That branch contains auto-sync merge commits or other history that doesn't exist in josh's filtered view
|
||||||
|
- Someone merges `main` into a feature/staging branch and then merges it back — the criss-cross parents confuse josh's mapping
|
||||||
|
|
||||||
|
**Automatic handling:** josh-sync automatically falls back to linearizing the history when the direct push fails. Regular commits are cherry-picked individually (preserving authorship and messages), while merge commits are squashed into single commits via `cherry-pick -m 1`. The PR notes when this fallback was used. See [ADR-011](adr/011-linearize-fallback-reverse.md).
|
||||||
|
|
||||||
|
**Prevention:** In josh-synced subrepos, prefer **squash merges** when merging long-lived branches (stage, develop) into the synced branch. Squash merges produce a single commit with no merge parents, which josh can always map. Regular feature branch merges (short-lived, no auto-sync history) are usually fine.
|
||||||
|
|
||||||
|
**Manual resolution (if automatic fallback also fails):** This indicates a more fundamental history issue. Options:
|
||||||
|
1. Cherry-pick the desired changes manually onto a clean branch and push through josh
|
||||||
|
2. Run `josh-sync reset <target>` to re-establish history (destructive — all subrepo clones must re-fetch)
|
||||||
|
|
||||||
|
#### Filter or path mismatch
|
||||||
|
|
||||||
|
Josh-proxy couldn't map the push due to an incorrect filter. Check josh-proxy logs, verify the `josh_filter` or `subfolder` in `.josh-sync.yml` is correct, and ensure the subfolder exists in the monorepo.
|
||||||
|
|
||||||
### Import PR shows "No changes"
|
### Import PR shows "No changes"
|
||||||
|
|
||||||
|
|||||||
10
lib/auth.sh
10
lib/auth.sh
@@ -28,6 +28,16 @@ subrepo_auth_url() {
|
|||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# ─── Monorepo Auth URL ─────────────────────────────────────────────
|
||||||
|
# Derives a push-capable HTTPS URL from MONOREPO_API.
|
||||||
|
# MONOREPO_API shape: https://<host>/api/v1/repos/<org/repo>
|
||||||
|
|
||||||
|
mono_auth_url() {
|
||||||
|
local api_host_path
|
||||||
|
api_host_path=$(echo "$MONOREPO_API" | sed 's|https://||; s|/api/v1/repos/|/|')
|
||||||
|
echo "https://${BOT_USER}:${GITEA_TOKEN}@${api_host_path}.git"
|
||||||
|
}
|
||||||
|
|
||||||
# ─── Remote Queries ─────────────────────────────────────────────────
|
# ─── Remote Queries ─────────────────────────────────────────────────
|
||||||
|
|
||||||
subrepo_ls_remote() {
|
subrepo_ls_remote() {
|
||||||
|
|||||||
@@ -16,6 +16,11 @@ parse_config() {
|
|||||||
local config_json
|
local config_json
|
||||||
config_json=$(yq -o json "$config_file") || die "Failed to parse ${config_file}"
|
config_json=$(yq -o json "$config_file") || die "Failed to parse ${config_file}"
|
||||||
|
|
||||||
|
# Schema version check (optional field — absent means v1)
|
||||||
|
local schema_version
|
||||||
|
schema_version=$(echo "$config_json" | jq -r '.schema_version // 1')
|
||||||
|
[ "$schema_version" = "1" ] || die "Unsupported schema_version ${schema_version} in ${config_file} (this version of josh-sync supports schema_version 1 — see CHANGELOG.md for migration instructions)"
|
||||||
|
|
||||||
# Export global values
|
# Export global values
|
||||||
export JOSH_PROXY_URL
|
export JOSH_PROXY_URL
|
||||||
JOSH_PROXY_URL=$(echo "$config_json" | jq -r '.josh.proxy_url')
|
JOSH_PROXY_URL=$(echo "$config_json" | jq -r '.josh.proxy_url')
|
||||||
|
|||||||
166
lib/sync.sh
166
lib/sync.sh
@@ -226,7 +226,9 @@ ${BOT_TRAILER}: filter-change/${mono_branch}/$(date -u +%Y-%m-%dT%H:%M:%SZ)")
|
|||||||
# ─── Reverse Sync: Subrepo → Monorepo ──────────────────────────────
|
# ─── Reverse Sync: Subrepo → Monorepo ──────────────────────────────
|
||||||
#
|
#
|
||||||
# Always creates a PR on the monorepo — never pushes directly.
|
# Always creates a PR on the monorepo — never pushes directly.
|
||||||
# Returns: skip | pr-created | josh-rejected
|
# Returns: skip | skip-dirty | pr-created | josh-rejected
|
||||||
|
# skip — trees identical, nothing to do
|
||||||
|
# skip-dirty — trees differ but no human commits found (don't update state)
|
||||||
|
|
||||||
reverse_sync() {
|
reverse_sync() {
|
||||||
local mono_branch="$SYNC_BRANCH_MONO"
|
local mono_branch="$SYNC_BRANCH_MONO"
|
||||||
@@ -238,6 +240,8 @@ reverse_sync() {
|
|||||||
|
|
||||||
log "INFO" "=== Reverse sync: subrepo/${subrepo_branch} → mono/${mono_branch} ==="
|
log "INFO" "=== Reverse sync: subrepo/${subrepo_branch} → mono/${mono_branch} ==="
|
||||||
|
|
||||||
|
local force_mode="${SYNC_REVERSE_FORCE:-0}"
|
||||||
|
|
||||||
# 1. Clone subrepo
|
# 1. Clone subrepo
|
||||||
git clone "$(subrepo_auth_url)" \
|
git clone "$(subrepo_auth_url)" \
|
||||||
--branch "$subrepo_branch" --single-branch \
|
--branch "$subrepo_branch" --single-branch \
|
||||||
@@ -265,16 +269,32 @@ reverse_sync() {
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
# 4. Find new human commits (excludes bot commits from forward sync)
|
# 4. Find new human commits (excludes bot commits from forward sync)
|
||||||
# Uses --ancestry-path to restrict to the direct lineage and avoid
|
# Uses merge-base + --ancestry-path to restrict to the direct lineage.
|
||||||
# leaking old history through reconciliation merge parents.
|
# merge-base is needed because mono-filtered may have advanced past the
|
||||||
local human_commits
|
# last forward sync (any monorepo commit, even outside the subfolder,
|
||||||
human_commits=$(git log --ancestry-path "mono-filtered/${mono_branch}..HEAD" \
|
# causes josh to create a new filtered commit). Using mono-filtered
|
||||||
--oneline --invert-grep --grep="^${BOT_TRAILER}:" 2>/dev/null || echo "")
|
# directly with --ancestry-path returns empty when it's not an ancestor
|
||||||
|
# of HEAD. merge-base always finds the last connected point.
|
||||||
|
local base human_commits
|
||||||
|
base=$(git merge-base "mono-filtered/${mono_branch}" HEAD 2>/dev/null || echo "")
|
||||||
|
|
||||||
|
if [ -n "$base" ]; then
|
||||||
|
human_commits=$(git log --ancestry-path "${base}..HEAD" \
|
||||||
|
--oneline --invert-grep --grep="^${BOT_TRAILER}:" 2>/dev/null || echo "")
|
||||||
|
else
|
||||||
|
# No common ancestor (first sync or fully diverged) — skip ancestry-path
|
||||||
|
human_commits=$(git log "mono-filtered/${mono_branch}..HEAD" \
|
||||||
|
--oneline --invert-grep --grep="^${BOT_TRAILER}:" 2>/dev/null || echo "")
|
||||||
|
fi
|
||||||
|
|
||||||
if [ -z "$human_commits" ]; then
|
if [ -z "$human_commits" ]; then
|
||||||
log "INFO" "No new human commits in subrepo — nothing to sync"
|
if [ "$force_mode" = "1" ]; then
|
||||||
echo "skip"
|
log "WARN" "No human commits found — force mode: continuing anyway"
|
||||||
return
|
else
|
||||||
|
log "INFO" "No new human commits in subrepo — nothing to sync"
|
||||||
|
echo "skip-dirty"
|
||||||
|
return
|
||||||
|
fi
|
||||||
fi
|
fi
|
||||||
|
|
||||||
log "INFO" "New human commits to sync:"
|
log "INFO" "New human commits to sync:"
|
||||||
@@ -284,12 +304,129 @@ reverse_sync() {
|
|||||||
local ts
|
local ts
|
||||||
ts=$(date +%Y%m%d-%H%M%S)
|
ts=$(date +%Y%m%d-%H%M%S)
|
||||||
local staging_branch="auto-sync/subrepo-${subrepo_branch}-${ts}"
|
local staging_branch="auto-sync/subrepo-${subrepo_branch}-${ts}"
|
||||||
|
local push_ref="HEAD"
|
||||||
|
local linearized=false
|
||||||
|
|
||||||
if git push -o "base=${mono_branch}" "$(josh_auth_url)" "HEAD:refs/heads/${staging_branch}"; then
|
if ! git push -o "base=${mono_branch}" "$(josh_auth_url)" "${push_ref}:refs/heads/${staging_branch}" 2>/dev/null; then
|
||||||
log "INFO" "Pushed to staging branch via josh: ${staging_branch}"
|
# Josh rejects pushes when the history contains merge commits whose
|
||||||
|
# parents it cannot map (e.g., merges of staging/feature branches that
|
||||||
|
# include auto-sync history). Fall back to linearizing: cherry-pick
|
||||||
|
# regular commits as-is, squash only the problematic merge commits.
|
||||||
|
# See ADR-011.
|
||||||
|
log "WARN" "Direct push failed — linearizing history (cherry-pick + squash merges)"
|
||||||
|
log "INFO" "─── linearize start ───"
|
||||||
|
|
||||||
# 6. Create PR on monorepo (NEVER direct push)
|
local original_head
|
||||||
|
original_head=$(git rev-parse HEAD)
|
||||||
|
|
||||||
|
git checkout -b "tmp-linearize-$$" "mono-filtered/${mono_branch}" >/dev/null 2>&1 \
|
||||||
|
|| die "Failed to create linearize branch"
|
||||||
|
|
||||||
|
# Reuse merge-base from step 4 (same reasoning: mono-filtered may have
|
||||||
|
# advanced past the last forward sync, breaking --ancestry-path).
|
||||||
|
local commit_shas
|
||||||
|
if [ -n "$base" ]; then
|
||||||
|
commit_shas=$(git log --ancestry-path "${base}..${original_head}" \
|
||||||
|
--reverse --format="%H" --invert-grep --grep="^${BOT_TRAILER}:" 2>/dev/null || echo "")
|
||||||
|
else
|
||||||
|
commit_shas=$(git log "mono-filtered/${mono_branch}..${original_head}" \
|
||||||
|
--reverse --format="%H" --invert-grep --grep="^${BOT_TRAILER}:" 2>/dev/null || echo "")
|
||||||
|
fi
|
||||||
|
|
||||||
|
local sha
|
||||||
|
for sha in $commit_shas; do
|
||||||
|
local parent_count
|
||||||
|
parent_count=$(git cat-file -p "$sha" | grep -c "^parent ")
|
||||||
|
local short_msg
|
||||||
|
short_msg=$(git log -1 --format="%s" "$sha")
|
||||||
|
|
||||||
|
if [ "$parent_count" -le 1 ]; then
|
||||||
|
# Regular commit — cherry-pick to preserve message and authorship
|
||||||
|
if git cherry-pick "$sha" >/dev/null 2>&1; then
|
||||||
|
log "INFO" " cherry-pick ${sha:0:7}: ${short_msg}"
|
||||||
|
else
|
||||||
|
log "WARN" " conflict ${sha:0:7} — applying patch from parent"
|
||||||
|
git cherry-pick --abort 2>/dev/null
|
||||||
|
local msg author_name author_email author_date
|
||||||
|
msg=$(git log -1 --format="%B" "$sha")
|
||||||
|
author_name=$(git log -1 --format="%an" "$sha")
|
||||||
|
author_email=$(git log -1 --format="%ae" "$sha")
|
||||||
|
author_date=$(git log -1 --format="%aI" "$sha")
|
||||||
|
# Diff from the commit's own parent — NOT from HEAD (which is
|
||||||
|
# the linearize branch and may have a different base).
|
||||||
|
git diff "${sha}^" "$sha" | git apply --index >/dev/null 2>&1 \
|
||||||
|
|| die "Failed to apply diff for ${sha:0:7}"
|
||||||
|
GIT_AUTHOR_NAME="$author_name" GIT_AUTHOR_EMAIL="$author_email" \
|
||||||
|
GIT_AUTHOR_DATE="$author_date" \
|
||||||
|
git commit -m "$msg" >/dev/null 2>&1 || die "Failed to commit ${sha:0:7}"
|
||||||
|
log "INFO" " applied ${sha:0:7}: ${short_msg}"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
# Merge commit — cherry-pick relative to first parent (squashes the merge)
|
||||||
|
local msg author_name author_email author_date
|
||||||
|
msg=$(git log -1 --format="%B" "$sha")
|
||||||
|
author_name=$(git log -1 --format="%an" "$sha")
|
||||||
|
author_email=$(git log -1 --format="%ae" "$sha")
|
||||||
|
author_date=$(git log -1 --format="%aI" "$sha")
|
||||||
|
|
||||||
|
if git cherry-pick -m 1 "$sha" >/dev/null 2>&1; then
|
||||||
|
log "INFO" " squash ${sha:0:7}: ${short_msg}"
|
||||||
|
else
|
||||||
|
# Cherry-pick failed — distinguish empty result (changes already
|
||||||
|
# applied by a prior cherry-pick) from a real conflict.
|
||||||
|
if git diff --cached --quiet 2>/dev/null && git diff --quiet 2>/dev/null; then
|
||||||
|
# Working tree clean — the merge changes are already present
|
||||||
|
git cherry-pick --abort 2>/dev/null || git reset --hard HEAD >/dev/null 2>&1
|
||||||
|
log "INFO" " skip ${sha:0:7}: ${short_msg} (already applied)"
|
||||||
|
else
|
||||||
|
# Real conflict — apply diff from first parent (what the merge
|
||||||
|
# actually introduced, NOT from HEAD which would capture all
|
||||||
|
# tree differences).
|
||||||
|
git cherry-pick --abort 2>/dev/null
|
||||||
|
git diff "${sha}^1" "$sha" | git apply --index >/dev/null 2>&1 \
|
||||||
|
|| die "Failed to apply merge diff for ${sha:0:7}"
|
||||||
|
GIT_AUTHOR_NAME="$author_name" GIT_AUTHOR_EMAIL="$author_email" \
|
||||||
|
GIT_AUTHOR_DATE="$author_date" \
|
||||||
|
git commit -m "$msg" >/dev/null 2>&1 || die "Failed to commit merge ${sha:0:7}"
|
||||||
|
log "INFO" " squash ${sha:0:7}: ${short_msg}"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
log "INFO" "─── linearize done ────"
|
||||||
|
|
||||||
|
push_ref="HEAD"
|
||||||
|
linearized=true
|
||||||
|
|
||||||
|
git push -o "base=${mono_branch}" "$(josh_auth_url)" \
|
||||||
|
"${push_ref}:refs/heads/${staging_branch}" \
|
||||||
|
|| { log "ERROR" "Josh rejected linearized push — check josh-proxy logs"; echo "josh-rejected"; return; }
|
||||||
|
fi
|
||||||
|
|
||||||
|
log "INFO" "Pushed to staging branch via josh: ${staging_branch}${linearized:+ (linearized)}"
|
||||||
|
|
||||||
|
# 6. Publish: force-push directly (--force) or create PR (default)
|
||||||
|
if [ "$force_mode" = "1" ]; then
|
||||||
|
log "WARN" "Force mode: pushing ${staging_branch} directly to mono/${mono_branch} — bypassing PR"
|
||||||
|
local mono_dir="${work_dir}/monorepo"
|
||||||
|
git clone "$(mono_auth_url)" \
|
||||||
|
--branch "$staging_branch" --single-branch \
|
||||||
|
"$mono_dir" || die "Failed to clone monorepo staging branch for force-push"
|
||||||
|
git -C "$mono_dir" push --force "$(mono_auth_url)" \
|
||||||
|
"HEAD:refs/heads/${mono_branch}" \
|
||||||
|
|| die "Failed to force-push to mono/${mono_branch}"
|
||||||
|
log "INFO" "Force-pushed mono/${staging_branch} → mono/${mono_branch}"
|
||||||
|
echo "forced"
|
||||||
|
else
|
||||||
local pr_body
|
local pr_body
|
||||||
|
local linearize_note=""
|
||||||
|
if [ "$linearized" = true ]; then
|
||||||
|
linearize_note="
|
||||||
|
> **Note:** Merge commits were squashed during sync because the subrepo
|
||||||
|
> history contained merges that josh-proxy cannot map (see ADR-011).
|
||||||
|
> Regular commits are preserved individually."
|
||||||
|
fi
|
||||||
pr_body="## Subrepo changes
|
pr_body="## Subrepo changes
|
||||||
|
|
||||||
New commits from subrepo \`${subrepo_branch}\`:
|
New commits from subrepo \`${subrepo_branch}\`:
|
||||||
@@ -297,7 +434,7 @@ New commits from subrepo \`${subrepo_branch}\`:
|
|||||||
\`\`\`
|
\`\`\`
|
||||||
${human_commits}
|
${human_commits}
|
||||||
\`\`\`
|
\`\`\`
|
||||||
|
${linearize_note}
|
||||||
**Review checklist:**
|
**Review checklist:**
|
||||||
- [ ] Changes scoped to synced subfolder
|
- [ ] Changes scoped to synced subfolder
|
||||||
- [ ] No leaked credentials or environment-specific config
|
- [ ] No leaked credentials or environment-specific config
|
||||||
@@ -311,9 +448,6 @@ ${human_commits}
|
|||||||
|
|
||||||
log "INFO" "Reverse sync PR created on monorepo"
|
log "INFO" "Reverse sync PR created on monorepo"
|
||||||
echo "pr-created"
|
echo "pr-created"
|
||||||
else
|
|
||||||
log "ERROR" "Josh rejected push — check josh-proxy logs"
|
|
||||||
echo "josh-rejected"
|
|
||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -5,6 +5,11 @@
|
|||||||
"type": "object",
|
"type": "object",
|
||||||
"required": ["josh", "targets", "bot"],
|
"required": ["josh", "targets", "bot"],
|
||||||
"properties": {
|
"properties": {
|
||||||
|
"schema_version": {
|
||||||
|
"type": "integer",
|
||||||
|
"const": 1,
|
||||||
|
"description": "Schema version. Bump when config fields are removed or renamed (matches josh-sync major version)."
|
||||||
|
},
|
||||||
"josh": {
|
"josh": {
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"required": ["proxy_url", "monorepo_path"],
|
"required": ["proxy_url", "monorepo_path"],
|
||||||
|
|||||||
Reference in New Issue
Block a user