Renderive/render_3D/datoviz/spec/api/PYTHON_GSP_SCOPE.md

Datoviz, Python Binding, GSP, and VisPy2 Scope

Status: normative v0.4 API and documentation positioning.

This document records the v0.4 ownership split between Datoviz, the generated Python binding, GSP, and VisPy2. It exists so release notes, public documentation, examples, and future binding work do not accidentally recreate the v0.3 Python API inside the Datoviz repository. The complementary backend strategy and capability-extension model lives in GSP_BACKEND_STRATEGY.md. The concrete RC-lane checklist for Datoviz as a GSP/Matplotlib rendering target lives in GSP_BACKEND_READINESS.md.

Decision

Datoviz v0.4 owns the native renderer/runtime foundation:

1. the C API;
2. retained scene/app objects;
3. DRP2 emission and runtime execution;
4. native Vulkan and experimental WebGPU/WASM backend work;
5. native interaction, cameras/controllers, animation, picking/probing, selection, capture, and performance-oriented examples;
6. one generated ctypes binding that honestly mirrors the intended v0.4 C surface.

GSP owns the backend-independent scientific-visualization rendering representation:

1. visuals;
2. data buffers;
3. transforms;
4. style/state needed to describe rendered visual output;
5. a renderer-neutral contract that can target Datoviz, Matplotlib, and future renderers.

VisPy2 owns the Pythonic user experience above GSP:

1. object-oriented scene and plotting APIs;
2. Python-level convenience constructors and data workflows;
3. interaction tools, camera abstractions, selections, animations, and application/notebook integration when those concepts are user-facing Python APIs;
4. backend selection and fallback behavior across Datoviz, Matplotlib, and other renderers.

GSP means Graphics Specification Protocol in the current Datoviz planning documents.

Rationale

The v0.3 Datoviz repository included a Pythonic object-oriented API. In v0.4, that API should not be rebuilt inside Datoviz if its shape is intended to match GSP. Keeping the Pythonic API in Datoviz would make one renderer repository the de facto owner of a backend-independent visual model, and would make Matplotlib or other renderers look like adapters to Datoviz rather than peer GSP renderers.

The cleaner split is:

Datoviz
  C renderer/runtime, scene/app, DRP2, native interaction, Python binding

GSP
  backend-independent visual rendering representation

VisPy2
  Pythonic plotting, scene objects, interaction tools, and backend selection

This lets Datoviz v0.4 release independently as an engine/API/architecture release. GSP and VisPy2 can then provide the Pythonic scientific visualization layer in a separate release cycle.

Core, Capability, And Extension Boundary

GSP should not become either a Datoviz clone or a lowest-common-denominator plotting API. The expected split is:

1. GSP core: portable semantic objects such as points, paths, images, meshes, axes, cameras, transforms, colormaps, labels, and layout intent;
2. GSP capabilities: portable requests whose quality or performance varies by backend, such as volumes, 3D interaction, picking/probing, large updates, transparency, and capture;
3. backend extensions: explicitly backend-specific features such as Datoviz PBR, ray tracing, custom shaders, advanced postprocess effects, and out-of-core residency controls.

Backend extensions must be discoverable through capability reporting and should degrade with clear warnings when a renderer cannot provide the requested behavior.

Direct Datoviz access remains the correct route for advanced renderer features, native embedding, high-throughput streaming, DRP2/runtime validation, custom visuals, and experimental work that is not yet portable through GSP.

Public Documentation Consequences

Datoviz documentation should be C-first and renderer-focused:

1. full C getting-started guide, tutorials, examples, visual reference, and advanced runtime topics;
2. Python binding documentation, including datoviz.raw exact calls for low-level integration and backend authors;
3. a clear "which API should I use?" page;
4. a Datoviz-as-GSP-backend page once the adapter is public;
5. links to GSP/VisPy2 for Pythonic plotting and high-level scientific workflows.

Datoviz documentation should not host the complete high-level VisPy2/GSP user guide. It may include short positioning notes and bridge documentation, but Pythonic tutorials, plotting examples, notebook workflows, backend-comparison examples, and high-level interaction docs should live with VisPy2/GSP.

The datoviz.raw exact-call documentation should set expectations explicitly:

1. it mirrors the C API;
2. object ownership follows the C API;
3. it is useful for tests, backend integration, and low-level automation;
4. it is not the recommended Pythonic plotting interface.

Agent-Facing Python Rules

When users ask a coding agent for Python code, Datoviz documentation should make the intended layer unambiguous:

1. use import datoviz as dvz for ordinary Python binding calls;
2. use datoviz.raw only when the user wants exact pointers, counts, bytes, callbacks, or ABI debugging;
3. preserve exact dvz_*, Dvz*, and DVZ_* names in Python examples;
4. keep datoviz.raw examples close to the C ownership model and destroy rules;
5. use narrow host helpers only for callback and event-loop ergonomics;
6. route high-level plotting, notebook workflows, and Pythonic scene objects to VisPy2/GSP;
7. do not invent datoviz.scatter(), datoviz.imshow(), or v0.3-style object APIs inside Datoviz v0.4 examples.

If a future thin Python scene helper is introduced, it must be documented as a specific supported surface with examples and validation. Until then, generated Python should not assume such helpers exist.

Examples Consequences

Datoviz should keep C examples as first-class release proof. They validate the actual v0.4 Datoviz surface: retained scene objects, app/window/offscreen paths, visual families, controllers, picking/probing, capture, DRP2/runtime behavior, and performance.

Datoviz should include only a small raw-ctypes Python example set, for example:

1. load the shared library and query the version;
2. create/destroy a minimal scene path;
3. render or capture a simple offscreen point/image scene when available;
4. run a tiny pick/probe smoke when runtime support is available.

Rich Python scientific workflows should live in VisPy2/GSP. Advanced example ownership should be decided by purpose:

1. renderer capability proof belongs in Datoviz C;
2. protocol/runtime validation belongs in Datoviz C or DRP2/DVZR fixtures;
3. Pythonic plotting, dashboard, notebook, and data-workflow examples belong in VisPy2/GSP;
4. backend comparison examples belong in VisPy2/GSP.

Interaction Components

GSP may remain render-only. User-facing Python interaction does not need to be part of the GSP core contract. VisPy2 can own the Python interaction layer above GSP: events, cameras, controllers, selection tools, animations, and application integration.

Datoviz can still provide reusable C interaction components for VisPy2 to wrap, provided those components are renderer-neutral. The desired boundary is:

VisPy2 events/tools
  -> reusable Datoviz controller/camera/math components
  -> GSP visual state or backend requests
  -> Datoviz, Matplotlib, or another renderer

Avoid this boundary:

VisPy2 interaction
  -> Datoviz scene/app internals
  -> Datoviz renderer only

Current Datoviz code is close but not fully exposed this way. The input module is already fairly standalone. Panzoom, arcball, fly, and turntable logic are mostly renderer-neutral internally, but their public constructors are scene-owned. Camera and animation are also currently panel/scene-owned from the public API. A future extraction pass should expose explicit standalone public constructors and state-export functions for reusable controller/camera components while keeping scene-owned Datoviz wrappers on top.

Release Positioning

Datoviz v0.4 may be released before GSP/VisPy2 is ready, as long as the public messaging is honest:

1. v0.4 is the native C renderer/runtime foundation;
2. the v0.3-style high-level Python object-oriented API is not part of Datoviz v0.4;
3. the generated Python binding is included for low-level integration and backend work;
4. the Pythonic scientific visualization API is expected to live in GSP/VisPy2;
5. GSP/VisPy2 follows on its own release timeline.

Public announcements should not imply that Datoviz v0.4 is the Pythonic successor to v0.3.