Google Drive Watchers

Google Drive watchers keep documents in one directly shared folder synchronized into a Synth project. They use a read-only Google service account, stable Drive file identity, durable change tokens, and periodic complete reconciliation.

Use a dedicated folder for the first rollout. Share only that folder with the service account, start with auto_start: false, run one manual sync, and inspect the resulting pages before enabling scheduled polling.

What Is Supported

Drive content	Synth materialization
Google Docs	Markdown export
Google Slides and Drawings	PDF export
PDF	Native download
DOCX	Native download
Markdown and text	Native download
Google Sheets	Not ingested in the first release
Shortcuts	Recorded as unsupported; targets are not followed

Folder traversal is recursive by default. A folder URL or bare folder ID is accepted. Link-only resource keys are not supported; share the folder directly with the service-account email instead.

1. Create A Dedicated Google Cloud Project

1. Create or select a Google Cloud project used only for the watcher.
2. Enable Google Drive API in that project.
3. Create a service account such as synth-drive-reader.
4. Create a JSON key for that service account and save it outside the Synth project and source repository.
5. Restrict the key file to the account running Synth (chmod 600 on Linux or macOS).

The watcher requests only the drive.readonly OAuth scope. It does not need Editor access to the folder.

2. Share The Folder

1. In Google Drive, create or choose the root folder to synchronize.
2. Open Share and add the service-account email from the JSON key.
3. Grant Viewer access.
4. Copy the folder URL. The value after /folders/ is the folder ID.

Sharing a parent workspace or relying on “Anyone with the link” is not a substitute for direct service-account access. Preview and the first sync will fail closed if the configured root is not an accessible folder.

3. Save The Credential In Synth

In Settings → Credentials, create a Personal Google Drive Service Account credential. An administrator may instead create a Global credential in Global Settings → Credentials when everyone should be able to select it. Paste the complete JSON key into the credential field, save it, and then delete any unnecessary working copy of the key.

Watcher configuration stores only the credential ID. At runtime Synth writes a watcher-scoped temporary key file with mode 0600, uses it to mint short-lived Google access tokens, and removes it when the watcher stops or a one-shot run finishes.

4. Configure With CaSC

Add the watcher to the project’s .synth/config.yaml:

version: 1
default_provider: openai
default_model: gpt-4o-mini
watchers:
  drive-docs:
    type: gdrive
    interval: 300
    auto_start: false
    credential: your-saved-credential-id
    config:
      folder_id: your-folder-id
      recursive: true
      follow_shortcuts: false
      max_file_bytes: 52428800
      max_batch_items: 200
      max_batch_bytes: 536870912
      max_scope_items: 100000
      full_reconcile_interval_seconds: 86400
      removal_policy: tombstone

Set default_provider and default_model to a provider already configured for the project. The API key remains in the provider environment or secret store; it does not belong in CaSC.

Keep credential at the watcher level. Do not put JSON, private keys, access tokens, credentials_file, or resource keys in CaSC. Invalid or unsafe Drive entries are ignored and surfaced as a configuration warning instead of being partially started.

For a small first rollout, reduce max_scope_items, max_batch_items, and max_batch_bytes. The minimum polling interval is 30 seconds; 300 seconds is a safer production default.

5. Preview And Run The First Sync

The Google Drive setup UI is enabled by default. Deployments that need an emergency rollout pause can set SYNTH_ENABLE_GDRIVE_WATCHER=0 in the Synth server environment and restart the server to hide it without deleting saved watcher configuration.

When Google Drive is enabled in the Watchers UI:

1. Open Watchers → New Watcher → Google Drive.
2. Select the saved service-account credential.
3. Paste the folder URL or ID.
4. Preview the scope. Preview is read-only: it checks root access, change-log access, recursive coverage, and supported content without advancing a checkpoint.
5. Save with auto-start disabled.
6. Select Run once, follow the logs, and inspect the created pages.
7. Start scheduled polling only after the watcher reports a committed first checkpoint and the pages contain the expected current content.

If the UI is not exposed by your deployment, the same watcher can be declared through CaSC and exercised with the watcher API.

Reading Drive Upload Logs

When a new or changed Drive file needs extraction, the watcher and ingestion worker report one continuous lifecycle. For example, uploading a PDF named Quarterly Report.pdf can produce:

[drive-docs] Scheduled check found 1 item
[drive-docs] Queued "Quarterly Report.pdf" for ingestion
[drive-docs] Sync complete: 4 pages, 4 connections from 1 item [ok] — "Quarterly Report.pdf"

The exact page and connection counts depend on the document. They describe knowledge created or refreshed by extraction, not the number of Drive files. A single PDF can produce several knowledge pages and connections.

Log or card signal	Meaning	Operator action
Scheduled check found 1 item	The Drive scan detected one supported new or changed file.	Keep following the current run.
Queued "Quarterly Report.pdf" for ingestion	The durable ingestion queue accepted that exact Drive file. This is an in-progress handoff, not a completed warning or success.	Wait for the terminal result; do not start a duplicate run.
Waiting for Google Drive ingestion worker	The watcher is waiting for extraction or recovering an already queued job.	Leave the watcher running and follow logs.
Sync complete: 4 pages, 4 connections from 1 item [ok] — "Quarterly Report.pdf"	Extraction, grounding validation, publication, and checkpoint finalization succeeded for the named file.	Inspect the resulting pages if this is a first sync or a high-impact document.
Drive checkpoint 12 committed and 4 live files in the trusted snapshot	The published knowledge and current Drive membership are durable; the watcher can safely advance from this generation.	No action when freshness is Current.
No new changes [ok]	The scheduled poll completed normally without extraction work.	No action.
Google Drive ingestion failed: ... or Blocked	The worker or publication path failed before a safe checkpoint commit.	Repair the reported cause, then retry or run Full reconcile.

On the normal path, the watcher records the terminal result as soon as the ingestion worker finishes; it does not wait for the next polling interval. Queue state and the active Drive batch are durable, so a process restart can recover the same job before processing later Drive changes.

For a multi-file batch, logs name up to three Drive files and summarize the remainder as + N more. Logs use Drive titles rather than opaque temporary artifact paths. Source titles are bounded and sanitized before display.

Watcher logs are operational history. Deploying a newer Synth version does not rewrite older entries, so logs created by an older release may still contain a generic Queued for ingestion worker warning. The current card ignores that legacy queue handoff when determining sync health. Use Clear only when you no longer need the older log history.

Freshness And Checkpoint Safety

The Drive file ID is the source identity. Renaming a file or moving it within the watched tree updates provenance without creating a new source identity.

Every synchronization follows checkpoint-last semantics:

1. Read the currently committed Drive change token.
2. Build a complete initial snapshot or drain the next change pages.
3. Materialize changed content into a watcher-scoped staging area.
4. Prepare and validate the knowledge mutation draft.
5. Publish pages, grounding, and live-source support.
6. Commit membership and the terminal Drive token only after publication succeeds.

If download, extraction, provider use, grounding validation, or publication fails, the batch becomes blocked and the Drive token does not advance. The next operator action can retry or run a full reconcile without silently skipping the failed version.

Full Reconcile

Change tokens are efficient, but they are not the only source of truth. Synth runs a bounded complete traversal at the configured reconcile interval and when the token is invalid, folder reachability changes, the scope config changes, or an operator requests repair.

For a directly shared My Drive folder, Synth takes this bounded complete snapshot on every poll. Google’s service-account change feed can omit edits made by the folder owner in this topology. For a Shared Drive, Synth uses the changes feed for low-latency updates and retains scheduled complete reconciliation as the repair path. The watcher status reports the active change_detection_mode so operators can verify which contract is in force.

A complete snapshot may repair missed additions, renames, and moves. Absence is handled conservatively: a previously live source must be absent from two matching complete snapshots before absence alone can retire it. Incomplete search, access loss, rate limiting, and permission errors never count as proof of deletion.

Use Full reconcile after changing recursive scope, moving large subtrees, restoring permissions, rotating a credential, or investigating a stale/blocked freshness state.

Operational States

State	Meaning	Action
Ready	Last terminal checkpoint is published and polling is healthy.	No action.
Syncing	Snapshot, change drain, queue handoff, extraction, or publication is active.	Follow logs; do not start a second run.
Blocked	A batch failed before checkpoint commit.	Repair the reported cause, then run full reconcile.
Stale	The last completed check is older than the polling tolerance.	Run once and inspect provider/API health.
Reconcile required	Token, scope, or reachability evidence needs a complete scan.	Run full reconcile.
Paused	Config is saved but scheduled polling is stopped.	Start only after preview and first-sync review.

Credential-Gated Acceptance Test

The Docker-first harness always runs deterministic fake-provider coverage. An optional live mode proves the real Drive REST contract and content exports while keeping secrets and Drive identifiers out of reports:

SYNTH_GDRIVE_LIVE=1 \
SYNTH_GDRIVE_CREDENTIAL_FILE=/absolute/path/service-account.json \
SYNTH_GDRIVE_FOLDER_ID=your-folder-id \
bash harness/scripts/harness_docker.sh \
  --harness gdrive-watcher \
  --runs 1 \
  --require-report

To test real knowledge extraction too, set SYNTH_GDRIVE_RUN_LLM=1 and provide SYNTH_PROVIDER plus SYNTH_PROVIDER_KEY_FILE. The provider key file must contain only the key and must remain outside the repository.

Troubleshooting

Symptom	Likely cause	Fix
Root access denied	Folder was not directly shared with the service account.	Share the root with the exact service-account email as Viewer.
Credential invalid	JSON is incomplete, revoked, or not a service-account key.	Create a new key, replace the saved credential, and reconcile.
Unsupported files	Sheets, shortcuts, or unrecognized binary types are in scope.	Move them out of scope or narrow MIME/extension filters.
Batch capacity exceeded	The scope or one batch exceeds configured item/byte bounds.	Narrow the scope or deliberately raise the relevant bound.
Ingestion blocked	LLM/provider, grounding, or draft publication failed.	Fix the provider or reported integrity error, then reconcile.
Filename appears as queued with no terminal result	Extraction is still running, the worker is retrying, or a restart recovery is in progress.	Follow logs and worker health; do not start another run while the card says Syncing.
Old generic queue warning remains in logs	The entry was retained from a release that recorded queue handoff as a warning.	Check the current card and latest terminal checkpoint; clear historical logs only if they are no longer needed.
Pages exceed changed file count	One document was decomposed into multiple knowledge pages.	Expected; inspect the cited source and generated pages if the count looks surprising.
No changes after rename	Content hash is unchanged.	Expected: metadata updates without redundant re-extraction.
Missing file is still live	Absence has only one complete-snapshot confirmation.	Run a second full reconcile after confirming the scope is correct.

Never troubleshoot by pasting the JSON key into logs, chat, issues, CaSC, or a shell command. Use a protected file or the Synth credential store.