This is the critical onboarding step. There are two approaches:
- **`josh-sync onboard`** (recommended) — interactive, resumable, preserves open PRs
- **Manual `import` → merge → `reset`** — lower-level, for automation or when there are no open PRs to preserve
### Option A: Onboard (recommended)
The `onboard` command walks you through the entire process interactively, with checkpoint/resume at every step.
**Before you start:**
1.**Rename** the existing subrepo on your Git server (e.g., `stores/storefront` → `stores/storefront-archived`)
2.**Create a new empty repo** at the original path (e.g., a new `stores/storefront` with no commits)
The rename preserves the archived repo with all its history and open PRs. The new empty repo will receive josh-filtered history.
**Run onboard:**
```bash
josh-sync onboard billing
```
The command will:
1.**Verify prerequisites** — checks the new empty repo is reachable, asks for the archived repo URL
2.**Import** — copies subrepo content into monorepo and creates import PRs (one per branch)
3.**Wait for merge** — shows PR numbers and waits for you to merge them
4.**Reset** — pushes josh-filtered history to the new subrepo (per-branch, with resume)
5.**Done** — prints instructions for developers and PR migration
If the process is interrupted at any point, re-run `josh-sync onboard billing` to resume from where it left off. Use `--restart` to start over.
**Migrate open PRs:**
After onboard completes, migrate PRs from the archived repo to the new one:
```bash
# Interactive — lists open PRs and lets you pick
josh-sync migrate-pr billing
# Migrate all open PRs at once
josh-sync migrate-pr billing --all
# Migrate specific PRs by number
josh-sync migrate-pr billing 5 8 12
```
PR migration works by fetching the diff from the archived repo's PR, applying it to the new repo, and creating a new PR. File content is identical after reset, so patches apply cleanly.
### Option B: Manual import → merge → reset
Use this when the subrepo has no open PRs to preserve, or for scripted automation.
> **You do NOT need to `git pull` locally before running reset.** The reset command clones fresh from josh-proxy — it never uses your local working copy.
This:
1. Clones the monorepo through josh-proxy with the josh filter (the "filtered view")
2. Force-pushes that filtered view to the subrepo, replacing its history
This establishes **shared commit ancestry** between josh's filtered view and the subrepo. Without this, josh-proxy can't compute diffs between the two.
> **Warning:** This is a destructive force-push that replaces the subrepo's history. Back up any important branches or tags in the subrepo beforehand. Merge or close all open pull requests on the subrepo first — they will be invalidated.
After reset, **every developer with a local clone of the subrepo** must update their local copy to match the new history:
```bash
cd /path/to/local-subrepo
git fetch origin
git checkout main && git reset --hard origin/main
git checkout stage && git reset --hard origin/stage # repeat for each branch
```
Or simply delete and re-clone the subrepo. Local-only branches (not pushed to the remote) will be lost either way.
| `SUBREPO_SSH_KEY` | SSH private key for subrepo push (if using SSH auth) |
| `SUBREPO_TOKEN` | Optional separate subrepo token (defaults to `SYNC_BOT_TOKEN`) |
> **GitHub Actions note:** These examples target Gitea Actions. For GitHub Actions, change the `uses:` reference to a GitHub repo (e.g., `org/josh-sync@v1`) and `runs-on:` to a GitHub runner (e.g., `ubuntu-latest`).
## How Ongoing Sync Works
Once set up, sync runs automatically:
### Forward sync (mono → subrepo)
Triggered by pushes to target subfolders or on a cron schedule:
1. Clones the monorepo through josh-proxy (filtered view of the subfolder)
2. Fetches the subrepo branch for comparison
3. If trees are identical → skip
4. If subrepo branch doesn't exist → fresh push
5. Merges mono changes on top of subrepo state
6. If clean merge → pushes with `--force-with-lease` (protects against concurrent changes)
7. If lease rejected → retries on next run (subrepo changed during sync)
8. If merge conflict → creates a conflict PR on the subrepo
### Reverse sync (subrepo → mono)
Runs on a cron schedule (never triggered by subrepo pushes):
1. Clones the subrepo
2. Fetches the monorepo's josh-filtered view for comparison
3. Finds new human commits (filters out bot commits by checking for the `Josh-Sync-Origin:` trailer)
4. If no new human commits → skip
5. Pushes through josh-proxy to a staging branch
6. Creates a PR on the monorepo — **never pushes directly**
### Loop prevention
Bot commits include a git trailer like `Josh-Sync-Origin: forward/main/2024-02-12T10:30:00Z`. Each sync direction filters out commits with this trailer, preventing changes from bouncing back and forth. The CI action also has a loop guard that skips entirely if the HEAD commit has the trailer.
### State tracking
Sync state is stored as JSON files on an orphan branch (`josh-sync-state`), one file per target/branch. This tracks the last-synced commit SHAs and timestamps to avoid re-syncing the same changes.
Some files in the monorepo subfolder may not belong in the subrepo (e.g., monorepo-specific CI configs, internal tooling). The `exclude` config field removes these at the josh-proxy layer — excluded files never appear in the subrepo.
When `exclude` is present, josh-sync generates a `.josh-filter-<target>.josh` file at the monorepo root containing a [josh stored filter](https://josh-project.github.io/josh/reference/filters.html) with `:exclude` clauses:
The file is referenced in the josh-proxy URL as `:+.josh-filter-<target>` (flat naming — josh-proxy's parser treats `/` in `:+` paths as a filter separator, so subdirectory paths don't work).
- Verify `monorepo_path` matches what josh-proxy expects
- Test manually: `git ls-remote https://<user>:<token>@josh.example.com/org/repo.git:/services/app.git`
### SSH authentication failures
-`SUBREPO_SSH_KEY` must contain the actual key content, not a file path
- For per-target keys, ensure `subrepo_ssh_key_var` in config matches the env var name
- Check the key has write access to the subrepo
### "Force-with-lease rejected"
Normal: the subrepo changed while sync was running. The next sync run will pick it up. If persistent, check for another process pushing to the subrepo simultaneously.
### "Josh rejected push" (reverse sync)
Josh-proxy couldn't map the push back to the monorepo. Check josh-proxy logs, verify the josh filter is correct. May indicate a history divergence — consider running `josh-sync reset <target>`.
### Import PR shows "No changes"
The subfolder already contains the same content as the subrepo. This is fine — the import is a no-op.
### Duplicate/looping commits
Verify `bot.trailer` in config matches what's in commit messages. Check the loop guard in the CI workflow is active.
**After reset (subrepo):** The subrepo's history was replaced by force-push. Local clones still have the old history:
```bash
cd /path/to/subrepo
git fetch origin
git checkout main && git reset --hard origin/main
```
Or simply delete and re-clone.
**After import/reset cycle (monorepo):** The import and reset steps create and update branches rapidly (`auto-sync/import-*`, `josh-sync-state`). If your local clone fetched partway through, tracking refs go stale:
