# Python Binding NumPy Adaptation

Status: active v0.4 Python binding direction; first generated NumPy-adaptation slice landed.

This note records the NumPy-adapted call form of the generated Python binding. Datoviz has one
`ctypes` binding. The goal is to make the normal `import datoviz as dvz` call form usable with
NumPy arrays while preserving the C-shaped Datoviz API and leaving high-level plotting to
GSP/VisPy2.

Related RC-lane GSP backend readiness work lives in
[`../api/GSP_BACKEND_READINESS.md`](../api/GSP_BACKEND_READINESS.md), especially the
`dvz_visual_set_data_many()` facade and offscreen capture-to-memory requirements.

Raw `ctypes` generation, skipped-function disposition, and FFI-helper policy are covered in
[CTYPES_POLICY.md](CTYPES_POLICY.md).


## Decision

The recommended direct-engine Python import should be:

```python
import datoviz as dvz
```

The top-level `datoviz` package should expose the normal Python call form over the C API:

1. preserve `dvz_*` function names;
2. accept NumPy arrays and compatible Python buffer objects for known data arguments;
3. infer pointer/count or pointer/byte-size arguments only where binding policy declares the
   relationship;
4. pass through unsupported or unannotated calls to the exact `datoviz.raw` call shape;
5. avoid prefixless aliases, object-oriented scene wrappers, and plotting functions.

The exact generated binding remains available as:

```python
import datoviz.raw as raw
```

`datoviz.raw` is for ABI validation, debugging, generator tests, advanced FFI work, and any call
site that needs exact `ctypes` arguments. The top-level package is the normal Python entry point for
direct Datoviz engine use.

Generated Python documentation examples should target the top-level package, not `datoviz.raw`, when
they are derived from canonical C examples. See
[EXAMPLE_PYTHON_GENERATION.md](EXAMPLE_PYTHON_GENERATION.md).


## Rationale

Datoviz v0.4 is C-first, but that should not imply that Python users must manually write
`ctypes.cast()` boilerplate for common array uploads. Scientists already have NumPy arrays. They
should be able to pass those arrays directly to Datoviz functions whose C contracts are known.

The top-level package should improve argument adaptation, not rename or remodel the API. Keeping `dvz_*` names
has several benefits:

1. C and Python examples map almost mechanically.
2. Headers, generated references, search results, and AI-assisted translations use the same names.
3. The Python binding remains visibly a direct engine API, not a half-designed plotting API.
4. Names such as `scatter`, `imshow`, `Figure`, and Pythonic visual objects remain available for
   GSP/VisPy2 or a future explicitly designed high-level layer.


## Public Layering

```text
datoviz
  Recommended direct-engine Python binding call form.
  Same dvz_* names as C.
  Accepts arrays for policy-declared pointer/count and pointer/byte-size argument groups.

datoviz.raw
  Exact generated ctypes call form.
  Requires explicit bytes, pointers, counts, and ctypes-compatible arguments.

GSP/VisPy2
  Future high-level plotting, Pythonic scene objects, notebook workflows, and scientific UX.
```

The public message should be:

> Datoviz can be used from Python today. The default Python package mirrors the C API and accepts
> NumPy arrays for common data arguments. It is not yet a high-level plotting package; that layer
> belongs above Datoviz in GSP/VisPy2.


## Example Shape

```python
import numpy as np
import datoviz as dvz

scene = dvz.dvz_scene()
figure = dvz.dvz_figure(scene, 512, 512, 0)
panel = dvz.dvz_panel_full(figure)

points = dvz.dvz_point(scene, 0)

positions = np.array(
    [
        [-0.5, -0.4, 0.0],
        [+0.5, -0.4, 0.0],
        [0.0, +0.5, 0.0],
    ],
    dtype=np.float32,
)
colors = np.array(
    [
        [255, 80, 80, 255],
        [80, 220, 120, 255],
        [90, 150, 255, 255],
    ],
    dtype=np.uint8,
)
diameters = np.array([18.0, 18.0, 18.0], dtype=np.float32)

dvz.dvz_visual_set_data(points, "position", positions)
dvz.dvz_visual_set_data(points, "color", colors)
dvz.dvz_visual_set_data(points, "diameter_px", diameters)

dvz.dvz_panel_add_visual(panel, points, None)
```

The corresponding raw calls remain available:

```python
import ctypes
import datoviz.raw as raw

raw.dvz_visual_set_data(
    points,
    b"position",
    positions.ctypes.data_as(ctypes.c_void_p),
    positions.shape[0],
)
```


## Generation Strategy

Do not hand-write wrappers for the whole API. Generate the facade from the same extracted metadata
used by the generated binding, plus source-controlled binding policy for ambiguous pointer relationships:

```text
C public headers
        |
        v
build/bindings/datoviz_api.json
        |
        +--> datoviz/_ctypes.py         generated ctypes implementation
        |
        +--> datoviz/_array_facade.py   generated NumPy adaptation
```

The generator should consume policy declarations from `spec/bindings/ctypes.yml` or a sibling
manifest. The policy should describe only relationships that cannot be inferred safely from C syntax:

1. pointer argument that accepts array-like data;
2. count argument inferred from `array.shape[0]`;
3. byte-size argument inferred from `array.nbytes`;
4. stride, shape, or dtype constraints when the C contract requires them;
5. string arguments that should accept Python `str` and encode to UTF-8 bytes;
6. ownership/lifetime rule, especially whether the callee copies data before returning.

Example policy shape:

```yaml
array_facade:
  dvz_visual_set_data:
    strings:
      - slot_name
    groups:
      - pointer_arg: data
        count_arg: item_count
        count_from: shape0
        lifetime: copied_before_return

  dvz_scene_buffer_set_data:
    groups:
      - pointer_arg: data
        size_arg: size_bytes
        size_from: nbytes
        lifetime: copied_before_return
```

The exact schema may change during implementation, but the policy must remain explicit. The
generator should not guess semantic pointer/count pairs blindly.


## Wrapper Behavior

For annotated functions, generated wrappers should:

1. accept NumPy arrays, memoryviews, and other compatible Python buffer objects where practical;
2. require or create C-contiguous arrays before passing a pointer;
3. infer item counts and byte sizes from the array object;
4. encode Python `str` values for configured `const char*` arguments;
5. keep converted temporary arrays alive until the raw call returns;
6. raise clear Python exceptions for unsupported dtype, shape, contiguity, or lifetime contracts;
7. return the raw function result without inventing ownership semantics.

For unannotated functions, the top-level package should expose the raw function directly or use a generated
trivial passthrough. This keeps the top-level namespace broad without pretending every pointer is
safe to adapt.


## Non-Goals

The NumPy-adapted call form must not provide:

1. prefixless aliases such as `scene()` or `visual_set_data()`;
2. Python scene, figure, panel, visual, or plot classes;
3. high-level constructors such as `scatter()`, `imshow()`, or `mesh()`;
4. automatic ownership or context-manager semantics for C handles;
5. implicit adaptation for unknown pointer arguments;
6. a compatibility layer for the v0.3 Python object model.


## Validation

The first implementation slice should include focused tests for:

1. importing `datoviz` and `datoviz.raw` with clear documented roles;
2. preserving `dvz_*` names in the top-level package;
3. Python `str` conversion for declared string arguments;
4. NumPy array conversion for pointer/count groups;
5. NumPy array conversion for pointer/byte-size groups;
6. non-contiguous input handling;
7. temporary lifetime during raw calls;
8. passthrough behavior for unannotated functions;
9. generated wrapper coverage reports from the policy manifest.

Suggested commands once implemented:

```sh
just ctypes
just ctypes-check
just ctypes-python-smoke
just ctypes-render-smoke
```


## Documentation Consequences

Public docs should not present Python support as two competing bindings. The accurate model is:

1. `datoviz` is the recommended direct-engine Python API and accepts NumPy arrays for declared data
   arguments;
2. `datoviz.raw` is the exact `ctypes` call form of the same generated binding;
3. neither layer is a high-level plotting API;
4. high-level Python scientific visualization belongs in GSP/VisPy2.

Raw examples may remain for ABI and low-level integration proof. User-facing Python examples should
prefer the top-level package.

Mechanically generated Python tabs for C examples should use the facade and policy described here;
the example-generation policy is recorded in
[EXAMPLE_PYTHON_GENERATION.md](EXAMPLE_PYTHON_GENERATION.md).