Computational Models

A model with a spatial component predicts the response to stimulation by a particular device, so it is bound to an implant and handed the stimulus. A temporal-only model describes one location’s response over time and needs no implant. Most users work with Model, which can contain a spatial component, a temporal component, or both:

SpatialModel determines where stimulation appears in the visual field.
TemporalModel determines how the response evolves over time.

Available models

Models are grouped by the tissue they stimulate. The root pulse2percept.models namespace holds only the abstract classes a model is assembled from and temporal models that are not tied to a stimulation site; models of a particular target live in pulse2percept.models.retina and pulse2percept.models.cortex.

Generic model components

Reference	Model	Type
generic	`FadingTemporal`	temporal
generic	`AlphaTemporal`	temporal

Both describe how one location’s response decays after a pulse, without committing to where that location is. They are the temporal half of a Model whose spatial half may be retinal or cortical.

Retinal stimulation

Reference	Model	Type
[Thompson2003]	`Thompson2003Model`	spatial
[Horsager2009]	`Horsager2009Model`	temporal
[Nanduri2012]	`Nanduri2012Model`	spatial + temporal
[Beyeler2019]	`ScoreboardModel`	spatial
[Beyeler2019]	`AxonMapModel`	spatial
derived from [Granley2021]	`BiphasicScoreboardModel`	spatiotemporal
[Granley2021]	`BiphasicAxonMapModel`	spatiotemporal

Every retinal spatial model derives from RetinalSpatial, which places electrodes through a retinotopic map and accepts a physical retinal extent as shorthand for xrange/yrange.

Which one to use depends on the scientific question. The three main choices differ in what they model:

ScoreboardModel: Fixed-width Gaussian per electrode; amplitude scales brightness.
BiphasicScoreboardModel: Adds [Granley2021]-derived pulse-dependent brightness and width. Requires a described biphasic pulse train rather than a bare amplitude.
BiphasicAxonMapModel: Additionally models axonal elongation, whose length follows phase duration [Granley2021].

The published models add assumptions specific to their experiments and should be chosen when those assumptions are relevant.

Cortical stimulation

Reference	Model	Type
[Beyeler2019], adapted	`ScoreboardModel`	spatial
[vanderGrinten2023]	`DynaphosModel`	spatiotemporal

Cortical spatial models derive from CortexSpatial, which simulates one or more visual areas (‘v1’, ‘v2’, ‘v3’) and maps them through cortical retinotopy. The cortical ScoreboardModel is a spatial baseline: it spreads current in cortex rather than in the retina, so phosphene size in the visual field follows cortical magnification. DynaphosModel adds the temporal dynamics of the [vanderGrinten2023] phosphene model, including charge accumulation and a stimulation threshold.

Basic usage

Models follow the same workflow: choose an implant, bind a model to it, then predict a percept from a stimulus.

import pulse2percept as p2p

implant = p2p.implants.retina.ArgusII()
model = p2p.models.retina.ScoreboardModel(implant=implant, rho=200)
percept = model.predict_percept({'A8': 30})

The result of predict_percept is a Percept.

Measuring a percept

Added in version 0.11.0.

Prediction stops at the percept. Measurement is optional post-processing, performed on call and not part of the model:

stimulus → model → Percept → optional measurement

measure() reports the brightness and geometry of the phosphenes in each frame:

percept = model.predict_percept({'A8': 30})
metrics = percept.measure()

metrics.peak.diameter               # dva
metrics.peak.centroid               # (x, y) in dva
metrics.peak.integrated_brightness  # brightness units x dva^2

Support

Each frame is clipped at zero, so negative model output does not contribute. Geometry is then measured on the support: the pixels at or above 50% of that frame’s own positive maximum. The threshold is relative and re-evaluated per frame, so scaling a percept’s brightness does not change its measured shape. Lower it to include more of the falloff:

metrics = percept.measure(threshold=0.25)

What is measured

max_brightness is the largest positive pixel value, in the model’s arbitrary units. integrated_brightness is the pixel sum scaled by pixel area (brightness units x dva^2), which approximates a spatial integral and is therefore insensitive to sampling resolution. Neither is on a psychophysical absolute scale.

The remaining measurements describe the support as a binary set of pixels; brightness enters only through where the threshold falls. For a sufficiently sampled circular phosphene, major_axis, minor_axis and diameter approximately agree.

area: Area of the support (dva^2).
diameter: Diameter (dva) of the circle of equal area. At threshold=0.5 this approximates the FWHM of a sufficiently sampled circular Gaussian.
centroid: Mean position (x, y) of the support pixels, in dva.
major_axis, minor_axis: Axes (dva) of the ellipse with the same second moments as the support.
elongation: major_axis / minor_axis; approaches 1 for a circular phosphene.
n_components: Number of disconnected suprathreshold regions.
touches_edge: Whether the support reaches the simulated field boundary.

A frame without positive brightness has no phosphene: brightness, area and component count are zero, and position and shape are NaN.

Multiple components and clipping

Model output need not form a single connected phosphene. n_components counts the disconnected suprathreshold regions; the largest is not selected, and every other measurement describes the combined support. An elongation of 4 is consistent with one elongated phosphene or with two round ones some distance apart, which n_components distinguishes.

touches_edge flags support reaching the border of the simulated visual field. Where it is True, the measurements describe only the portion inside the field, and no extrapolation is performed. Widen the model’s xrange and yrange to capture the full percept.

Temporal percepts

Frames are measured independently, and each measurement is also available as an array over frames:

metrics = percept.measure()

metrics.integrated_brightness  # one value per frame
metrics.peak_frame             # index of the brightest frame
metrics.peak                   # that frame's measurements

peak_frame ranks frames by integrated positive brightness, taking the earliest on a tie. Frame timing remains in percept.time; no time integration or duration metrics are computed.

Scope

These measurements describe the modeled percept image. They do not estimate visual acuity, phosphene discriminability, pairwise separability, behavioral resolution, or object-recognition performance.

measure() applies to model-produced brightness percepts and raises for the RGB percepts returned by scene composition, whose values are display intensities. Positions and sizes are in degrees of visual angle, so the percept must have been built on a Grid2D; a percept holding bare pixel indices is rejected.

Source, delivered stimulation, percept

The prediction pipeline distinguishes the source, delivered stimulation, and percept:

source → implant → delivered stimulation → model → percept

Source: Input presented to the device: a Stimulus (or compatible scalar, array, or dict), ImageStimulus, VideoStimulus, or Scene.
Delivered stimulation: Electrical stimulation after implant preprocessing, encoding, raster scheduling, threshold calibration, and safety checks. Models call implant.prepare_stim(source) internally; call it directly to inspect the delivered stimulus.
Percept: Model output from model.predict_percept(source).

Building

Models build automatically on first prediction. Changing a model parameter invalidates the affected component, which is rebuilt when needed:

model = p2p.models.retina.AxonMapModel(implant=implant)

# Builds automatically:
percept = model.predict_percept(stim)

# Rebuilds the spatial component:
model.spatial.rho = 250
percept = model.predict_percept(stim)

Rebinding the implant also invalidates the spatial build because it depends on device geometry. model.build() forces a full rebuild; to set parameters as you build a component, use model.spatial.build(rho=250).

Electrode-retina distance

ScoreboardModel, AxonMapModel and Thompson2003Model use electrode x and y coordinates only. Nonzero z values therefore do not affect their output and produce a warning.

This is a model limitation. Electrode-target distance is expected to affect stimulation threshold and spatial recruitment, but pulse2percept does not currently parameterize that relationship because the required psychophysical evidence is insufficient. In the Scoreboard and AxonMap models, rho remains an effective perceptual spread parameter fitted to subject reports rather than inferred from electrode-retina distance.

Simulating a visual scene

Added in version 0.11.0.

The workflow above starts from a stimulus you built yourself. To start from what someone is looking at instead, give the model a Scene. A scene is one monocular visual field, not the person’s final vision: it says what is present in front of one eye, and where that eye’s native vision is lost.

from pulse2percept.units import dva

scene = p2p.vision.Scene(p2p.stimuli.samples.logo_bvl(), fov=40 * dva)

implant = p2p.implants.retina.ArgusII()
implant.encoder = p2p.stimuli.AmplitudeEncoder(amp_range=(0, 50))

model = p2p.models.retina.ScoreboardModel(implant=implant, rho=200)
percept = model.predict_percept(scene, gaze=(0, 0) * dva)

Scene prediction separates four responsibilities:

Scene	What is visually present, and where native vision is lost.
Implant	Device geometry and encoding constraints.
Model	Knows the retinotopy, and so connects Scene to Implant.
Percept	What the simulated observer sees.

The model maps implant coordinates into the visual field through its retinotopic map. Each electrode follows this chain:

retinal coordinate (um)
  -> visual_field_map.ret_to_dva -> eye-centered visual field (dva)
  -> + gaze, for eye-coupled input only -> scene coordinate (dva)
  -> sample the scene

gaze is the scene location that currently falls on the fovea, so scene = eye-centered visual field + gaze. Gaze always decides where the percept lands in scene coordinates. Whether it also decides what the electrodes are given depends on the implant’s scene_input_frame:

`'eye'`	Input passes through the eye’s optics (Alpha, PRIMA), so gaze moves the scene across the implant as well as moving the percept across the scene. This is the default.
`'head'`	Input comes from a head-fixed camera (Argus, BVT, IMIE) that the eye cannot move: the electrodes are handed the same scene whatever the gaze, and only the percept moves.

Neither the implant nor an eye-centered Scotoma moves when gaze does. Pass one (x, y) to fixate, or one per video frame to move the eye between frames.

For a 'head' system, the sampling locations above are still the electrodes’ own visual-field positions, which assumes the device’s camera-to-electrode registration is aligned with them. Real systems configure that mapping separately and it is not modeled here.

scene_input_frame follows the device class but is a property of the system, so one implant can be run the other way – an Argus II with eye tracking, which shifts the camera ROI with gaze:

implant = p2p.implants.retina.ArgusII()
implant.scene_input_frame = 'eye'

The sampled values are passed to implant.encoder, which maps gray levels to current and applies device timing constraints. A scene is per-prediction input and is not stored on the model or implant.

An implant’s preprocess – an edge filter, an inversion, a contrast stretch – is applied to the prosthetic input branch only, before the scene is sampled at the electrode locations, because an image operation needs an image and by sampling time there is one number per electrode. Native and residual vision always use the original scene: what the device does to its own input is not something the eye goes through. Spatial preprocessing operates at the scene source’s pixel resolution.

implant.preprocess = lambda stim: stim.filter('sobel')

For scene input, preprocess must return an ImageStimulus or VideoStimulus; conversion to electrical stimulation belongs to the encoder. Pixel values and channels may change, but spatial shape and frame timing must remain unchanged because fov and the frame clock refer to the original scene.

Scene registration is a spatial-model capability: a model has to say where in the visual field each of its electrodes lands. Only retinal models (RetinalSpatial) implement it, through their retinotopy; any other spatial model raises NotImplementedError. A retinal model given a non-retinotopic visual_field_map, or an implant without an encoder, raises ValueError.

Residual vision

If the scene also carries a Scotoma, the result is what the person actually sees – intact native vision outside the lost region, and the prosthetic percept inside it – as a single RGB Percept on the scene’s own pixel grid:

scene = p2p.vision.Scene(p2p.stimuli.samples.logo_bvl(), fov=40 * dva,
                         scotoma=p2p.vision.Scotoma.circle(8 * dva))
model = p2p.models.retina.ScoreboardModel(implant=implant, rho=200)

percept = model.predict_percept(scene, gaze=(0, 0) * dva, vmax=50)

vmax is required here and is not inferred: model brightness is in arbitrary units, so which brightness counts as white is a claim about the display, not about the model. Holding it fixed across calls is what keeps two gazes comparable.

The scotoma affects native vision only. Prosthetic encoding samples the unmasked scene, including locations inside the scotoma.

The rendered field boundary

Added in version 0.11.0.

A scene’s source, pixel grid and sampling are rectangular. aperture='circle' renders an eye-centered disc of radius min(fov) / 2 instead, blacking out the corners around it and changing the rendered scene only:

scene = p2p.vision.Scene(image, fov=40 * dva, aperture='circle')

Like the scotoma and the eccentricity rings, the disc is eye-centered, so gaze moves it through the scene.

Both eyes

Added in version 0.11.0.

BinocularScene holds the left and right monocular views:

binocular = p2p.vision.BinocularScene(
    left=p2p.vision.Scene(image, fov=40 * dva, scotoma=scotoma),
    right=p2p.vision.Scene(image, fov=40 * dva),
)

ax_left, ax_right = binocular.plot(left_percept=percept, vmax=2)

A bilateral loss is often symmetric about the vertical meridian. mirror() reflects a scotoma across it (mirrored(x, y) == original(-x, y)) and returns a new one:

left_scotoma = p2p.vision.Scotoma.circle(3 * dva, center=(6, 0) * dva)
right_scotoma = left_scotoma.mirror()

Models are monocular in v0.11, so a prediction names the eye it is about:

percept = model.predict_percept(binocular.left)

Percept data layouts

A Percept holds one of two layouts, with time as the last axis in both:

(Y, X, T)     perceived brightness in arbitrary units
(Y, X, 3, T)  RGB intensities in [0, 1]

Prosthesis models produce brightness percepts. When a Scene has a scotoma, scene-driven prediction composes that model output with residual vision and returns an RGB percept:

import numpy as np
from pulse2percept.percepts import Percept

rgb = Percept(np.zeros((60, 80, 3, 1)))
rgb.is_rgb                  # True
rgb[..., 0].shape           # (60, 80, 3): one frame, still in color
rgb.plot()                  # drawn as RGB, without a colormap

RGB values are display intensities and must be finite and lie in [0, 1]; anything else raises at construction rather than saturating quietly later. The RGB axis is not a spatial dimension: space still describes (Y, X).

Operations defined on perceived brightness (i.e., n_gray, argmax, max, vmin, vmax) raise a ValueError for an RGB percept rather than inventing a conversion from color to brightness. Ranking three channels by one number would have to pick a color metric, which is also why a multi-frame RGB percept has no brightest frame to plot(); animate it with play() instead. percept.data is always available for the plain numerical answer.

Spatial and temporal components

Classes ending in Model are complete models with explicit constructor parameters:

model = p2p.models.retina.AxonMapModel(
    implant,
    rho=300,
    lam=500,
)
percept = model.predict_percept(stim)

Classes ending in Spatial or Temporal are components, and Model combines two of them:

spatial = p2p.models.retina.AxonMapSpatial(
    implant,
    rho=300,
    lam=500,
)

temporal = p2p.models.FadingTemporal(tau=100)

model = p2p.models.Model(spatial, temporal)

Use Model to combine spatial and temporal components from different models. At least one component is required, and each must already be constructed. The implant belongs to the spatial component.

Parameters

Component parameters are accessed directly:

model.spatial.rho = 250
model.temporal.tau = 50

Named-model constructors expose the same parameters directly. After construction, access them through the component. Parameters declared by both components, such as thresh_percept, remain independent.

The API reference for each model documents its assumptions, parameters, input requirements, and numerical units.