clone_catalog
Create an ML workspace by cloning catalog data reachable from a root RID, including schema, assets, and foreign keys with orphan handling.
Instructions
Create an ML workspace by cloning data reachable from a root RID.
Creates a partial catalog clone containing only data reachable from the root RID (e.g., a project, dataset, or experiment). Uses the root table's export annotation (if available) to determine which tables and paths to follow, then fills in any uncovered tables (vocabularies, associations).
Uses a three-stage approach:
Create schema WITHOUT foreign keys (only for included tables)
Copy data asynchronously (export paths + fill-in tables)
Apply foreign keys, handling violations based on orphan_strategy
Asset handling modes:
"none": Don't copy assets (asset columns will be empty)
"refs": Copy asset URLs only, files stay on source server (default)
"full": Download and re-upload all assets (fully independent clone)
Orphan handling: When source catalog policies hide some data but not references to it, cloning can result in dangling foreign keys. The orphan_strategy controls how these are handled.
Args: source_hostname: Source server hostname (e.g., "www.facebase.org"). source_catalog_id: ID of the catalog to clone. root_rid: The starting RID from which to trace reachability (e.g., a project RID like "3-HXMC"). dest_hostname: Destination hostname. If None, uses source hostname. alias: Optional alias name for the new catalog. add_ml_schema: If True, add the DerivaML schema to the clone. asset_mode: How to handle assets: "none", "refs" (default), or "full". copy_annotations: If True (default), copy all annotations. copy_policy: If True (default), copy ACL policies. exclude_schemas: List of schema names to exclude from cloning. exclude_objects: List of tables ("schema:table" format) to exclude. reinitialize_dataset_versions: If True (default), reinitialize dataset versions. orphan_strategy: How to handle orphan rows: "fail", "delete", or "nullify". prune_hidden_fkeys: If True, skip FKs with hidden reference data. truncate_oversized: If True, truncate values exceeding index size limits. include_tables: Additional tables to include. include_associations: If True, auto-include association tables. include_vocabularies: If True, auto-include vocabulary tables. table_concurrency: Max concurrent table copies during fill phase. Lower values reduce server load. Default: 1.
Returns: JSON with status, source info, destination info, and operation details including tables restored and orphan handling stats.
Examples: clone_catalog("www.facebase.org", "1", root_rid="3-HXMC", dest_hostname="localhost", alias="facebase-musmorph", add_ml_schema=True, orphan_strategy="delete")
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| alias | No | ||
| root_rid | Yes | ||
| asset_mode | No | refs | |
| copy_policy | No | ||
| add_ml_schema | No | ||
| dest_hostname | No | ||
| include_tables | No | ||
| exclude_objects | No | ||
| exclude_schemas | No | ||
| orphan_strategy | No | fail | |
| source_hostname | Yes | ||
| copy_annotations | No | ||
| source_catalog_id | Yes | ||
| table_concurrency | No | ||
| prune_hidden_fkeys | No | ||
| truncate_oversized | No | ||
| include_associations | No | ||
| include_vocabularies | No | ||
| reinitialize_dataset_versions | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |