pulse2percept.stimuli.videos

VideoStimulus, BostonTrain

Classes

BostonTrain([resize, electrodes, as_gray, …]) Boston Train sequence
VideoStimulus(source[, format, resize, …]) A stimulus made from a movie file, where each pixel gets assigned to an electrode, and grayscale values in the range [0, 255] get assigned to activation values in the range [0, 1].
class pulse2percept.stimuli.videos.BostonTrain(resize=None, electrodes=None, as_gray=False, metadata=None)[source]

Boston Train sequence

Load the Boston subway sequence, consisting of 94 frames of 240x426x3 pixels each.

New in version 0.7.

Parameters:
  • resize ((height, width) or None) – A tuple specifying the desired height and the width of the video stimulus.
  • electrodes (int, string or list thereof; optional, default: None) –

    Optionally, you can provide your own electrode names. If none are given, electrode names will be numbered 0..N.

    Note

    The number of electrode names provided must match the number of pixels in the (resized) video frame.

  • as_gray (bool, optional) – Flag whether to convert the image to grayscale. A four-channel image is interpreted as RGBA (e.g., a PNG), and the alpha channel will be blended with the color black.
  • metadata (dict, optional, default: None) – Additional stimulus metadata can be stored in a dictionary.
append(other)[source]

Append another stimulus

This method appends another stimulus (with matching electrodes) in time. The combined stimulus duration will be the sum of the two individual stimuli.

New in version 0.7.

Parameters:other (Stimulus) – Another stimulus with matching electrodes.
Returns:comb (Stimulus) – A combined stimulus with the same number of electrodes and new stimulus duration equal to the sum of the two individual stimuli.
apply(func, *args, **kwargs)[source]

Apply a function to each frame of the video

Parameters:
  • func (function) – The function to apply to each frame in the video. Must accept a 2D or 3D image and return an image with the same dimensions
  • *args – Additional positional arguments passed to the function
  • **kwargs – Additional keyword arguments passed to the function
Returns:

stim (ImageStimulus) – A copy of the stimulus object with the new image

center(loc=None)[source]

Center the image foreground

This function shifts the center of mass (CoM) to the image center.

Parameters:loc ((col, row), optional) – The pixel location at which to center the CoM. By default, shifts the CoM to the image center.
Returns:stim (ImageStimulus) – A copy of the stimulus object containing the centered image
compress()[source]

Compress the source data

Returns:compressed (Stimulus)
data

Stimulus data container A 2-D NumPy array that contains the stimulus data, where the rows denote electrodes and the columns denote points in time.

dt

Sampling time step (ms)

Defines the duration of the signal edge transitions.

New in version 0.7.

electrodes

Electrode names A list of electrode names, corresponding to the rows in the data container.

filter(filt, **kwargs)[source]

Filter each frame of the video

Parameters:
  • filt (str) –

    Image filter that will be applied to every frame of the video. Additional parameters can be passed as keyword arguments. The following filters are supported:

    • ’sobel’: Edge filter the image using the Sobel filter.
    • ’scharr’: Edge filter the image using the Scarr filter.
    • ’canny’: Edge filter the image using the Canny algorithm. You can also specify sigma, low_threshold, high_threshold, mask, and use_quantiles.
    • ’median’: Return local median of the image.
  • **kwargs – Additional parameters passed to the filter
Returns:

stim (VideoStimulus) – A copy of the stimulus object with the filtered image

invert()[source]

Invert the gray levels of the video

Returns:stim (VideoStimulus) – A copy of the stimulus object with all grayscale values inverted in the range [0, 1].
is_charge_balanced

Flag indicating whether the stimulus is charge-balanced

A stimulus with a time component is considered charge-balanced if its net current is smaller than 10 pico Amps

is_compressed

Flag indicating whether the stimulus has been compressed

play(fps=None, repeat=True, annotate_time=True, ax=None)[source]

Animate the percept as HTML with JavaScript

The percept will be played in an interactive player in IPython or Jupyter Notebook.

Parameters:
  • fps (float or None) – If None, uses the percept’s time axis. Not supported for non-homogeneous time axis.
  • repeat (bool, optional) – Whether the animation should repeat when the sequence of frames is completed.
  • annotate_time (bool, optional) – If True, the time of the frame will be shown as t = X ms in the title of the panel.
  • ax (matplotlib.axes.AxesSubplot, optional) – A Matplotlib axes object. If None, will create a new Axes object
Returns:

ani (matplotlib.animation.FuncAnimation) – A Matplotlib animation object that will play the percept frame-by-frame.

plot(electrodes=None, time=None, fmt='k-', ax=None)[source]

Plot the stimulus

New in version 0.7.

Parameters:
  • electrodes (int, string, or list thereof; optional, default: None) – The electrodes for which to plot the stimulus. If None, all electrodes are plotted.
  • time ((t_min, t_max) tuple, slice, or list of exact time points) – The time points at which to plot the stimulus. Specify a range of time points with a tuple or a slice, or specify the exact time points to interpolate. If None, all time points are plotted.
  • fmt (str, optional, default: 'k-') – A Matplotlib format string; e.g., ‘ro’ for red circles.
  • ax (matplotlib.axes.Axes or list thereof; optional, default: None) – A Matplotlib Axes object or a list thereof (one per electrode to plot). If None, a new Axes object will be created.
Returns:

axes (matplotlib.axes.Axes or np.ndarray of them) – Returns one matplotlib.axes.Axes per electrode

resize(shape, electrodes=None)[source]

Resize the video

Parameters:
  • shape ((rows, cols)) – Shape of each frame in the resized video. If one of the dimensions is set to -1, its value will be inferred by keeping a constant aspect ratio.
  • electrodes (int, string or list thereof; optional) –

    Optionally, you can provide your own electrode names. If none are given, electrode names will be numbered 0..N.

    Note

    The number of electrode names provided must match the number of pixels in the resized video.

Returns:

stim (VideoStimulus) – A copy of the stimulus object containing the resized video

rewind()[source]

Rewind the iterator

rgb2gray(electrodes=None)[source]

Convert the video to grayscale

Parameters:electrodes (int, string or list thereof; optional) –

Optionally, you can provide your own electrode names. If none are given, electrode names will be numbered 0..N.

Note

The number of electrode names provided must match the number of pixels in the grayscale video.

Returns:stim (VideoStimulus) – A copy of the stimulus object with all RGB values converted to grayscale in the range [0, 1].
rotate(angle, mode='constant')[source]

Rotate each frame of the video

Parameters:angle (float) – Angle by which to rotate each video frame (degrees). Positive: counter-clockwise, negative: clockwise
Returns:stim (VideoStimulus) – A copy of the stimulus object containing the rotated video
scale(scaling_factor)[source]

Scale the image foreground

This function scales the image foreground (excluding black pixels) by a factor.

Parameters:scaling_factor (float) – Factory by which to scale the image
Returns:stim (ImageStimulus) – A copy of the stimulus object containing the scaled image
shape

Data container shape

shift(shift_cols, shift_rows)[source]

Shift the image foreground

This function shifts the center of mass (CoM) of the image by the specified number of rows and columns.

Parameters:
  • shift_cols (float) – Number of columns by which to shift the CoM. Positive: to the right, negative: to the left
  • shift_rows (float) – Number of rows by which to shift the CoM. Positive: downward, negative: upward
Returns:

stim (ImageStimulus) – A copy of the stimulus object containing the shifted image

time

Time steps A list of time steps, corresponding to the columns in the data container.

class pulse2percept.stimuli.videos.VideoStimulus(source, format=None, resize=None, as_gray=False, electrodes=None, time=None, metadata=None, compress=False)[source]

A stimulus made from a movie file, where each pixel gets assigned to an electrode, and grayscale values in the range [0, 255] get assigned to activation values in the range [0, 1].

The frame rate of the movie is used to infer the time points at which to stimulate.

New in version 0.7.

Parameters:
  • source (str) –

    Path to video file. Supported file types include MP4, AVI, MOV, and GIF; and are inferred from the file ending. If the file does not have a proper file ending, specify the file type via format.

    Alternatively, pass a <rows x columns x channels x frames> NumPy array or another VideoStimulus object.

  • format (str) – A video format string supported by imageio, such as ‘MP4’, ‘AVI’, or ‘MOV’. Use if the file type cannot be inferred from source. For a full list of supported formats, see https://imageio.readthedocs.io/en/stable/formats.html.
  • resize ((height, width) or None, optional, default: None) – A tuple specifying the desired height and the width of each video frame
  • as_gray (bool, optional) – Flag whether to convert the image to grayscale. A four-channel image is interpreted as RGBA (e.g., a PNG), and the alpha channel will be blended with the color black.
  • electrodes (int, string or list thereof; optional, default: None) –

    Optionally, you can provide your own electrode names. If none are given, electrode names will be numbered 0..N.

    Note

    The number of electrode names provided must match the number of pixels in the (resized) image.

  • metadata (dict, optional, default: None) – Additional stimulus metadata can be stored in a dictionary.
  • compress (bool, optional, default: False) – If True, will compress the source data in two ways: * Remove electrodes with all-zero activation. * Retain only the time points at which the stimulus changes.
append(other)[source]

Append another stimulus

This method appends another stimulus (with matching electrodes) in time. The combined stimulus duration will be the sum of the two individual stimuli.

New in version 0.7.

Parameters:other (Stimulus) – Another stimulus with matching electrodes.
Returns:comb (Stimulus) – A combined stimulus with the same number of electrodes and new stimulus duration equal to the sum of the two individual stimuli.
apply(func, *args, **kwargs)[source]

Apply a function to each frame of the video

Parameters:
  • func (function) – The function to apply to each frame in the video. Must accept a 2D or 3D image and return an image with the same dimensions
  • *args – Additional positional arguments passed to the function
  • **kwargs – Additional keyword arguments passed to the function
Returns:

stim (ImageStimulus) – A copy of the stimulus object with the new image

center(loc=None)[source]

Center the image foreground

This function shifts the center of mass (CoM) to the image center.

Parameters:loc ((col, row), optional) – The pixel location at which to center the CoM. By default, shifts the CoM to the image center.
Returns:stim (ImageStimulus) – A copy of the stimulus object containing the centered image
compress()[source]

Compress the source data

Returns:compressed (Stimulus)
data

Stimulus data container A 2-D NumPy array that contains the stimulus data, where the rows denote electrodes and the columns denote points in time.

dt

Sampling time step (ms)

Defines the duration of the signal edge transitions.

New in version 0.7.

electrodes

Electrode names A list of electrode names, corresponding to the rows in the data container.

filter(filt, **kwargs)[source]

Filter each frame of the video

Parameters:
  • filt (str) –

    Image filter that will be applied to every frame of the video. Additional parameters can be passed as keyword arguments. The following filters are supported:

    • ’sobel’: Edge filter the image using the Sobel filter.
    • ’scharr’: Edge filter the image using the Scarr filter.
    • ’canny’: Edge filter the image using the Canny algorithm. You can also specify sigma, low_threshold, high_threshold, mask, and use_quantiles.
    • ’median’: Return local median of the image.
  • **kwargs – Additional parameters passed to the filter
Returns:

stim (VideoStimulus) – A copy of the stimulus object with the filtered image

invert()[source]

Invert the gray levels of the video

Returns:stim (VideoStimulus) – A copy of the stimulus object with all grayscale values inverted in the range [0, 1].
is_charge_balanced

Flag indicating whether the stimulus is charge-balanced

A stimulus with a time component is considered charge-balanced if its net current is smaller than 10 pico Amps

is_compressed

Flag indicating whether the stimulus has been compressed

play(fps=None, repeat=True, annotate_time=True, ax=None)[source]

Animate the percept as HTML with JavaScript

The percept will be played in an interactive player in IPython or Jupyter Notebook.

Parameters:
  • fps (float or None) – If None, uses the percept’s time axis. Not supported for non-homogeneous time axis.
  • repeat (bool, optional) – Whether the animation should repeat when the sequence of frames is completed.
  • annotate_time (bool, optional) – If True, the time of the frame will be shown as t = X ms in the title of the panel.
  • ax (matplotlib.axes.AxesSubplot, optional) – A Matplotlib axes object. If None, will create a new Axes object
Returns:

ani (matplotlib.animation.FuncAnimation) – A Matplotlib animation object that will play the percept frame-by-frame.

plot(electrodes=None, time=None, fmt='k-', ax=None)[source]

Plot the stimulus

New in version 0.7.

Parameters:
  • electrodes (int, string, or list thereof; optional, default: None) – The electrodes for which to plot the stimulus. If None, all electrodes are plotted.
  • time ((t_min, t_max) tuple, slice, or list of exact time points) – The time points at which to plot the stimulus. Specify a range of time points with a tuple or a slice, or specify the exact time points to interpolate. If None, all time points are plotted.
  • fmt (str, optional, default: 'k-') – A Matplotlib format string; e.g., ‘ro’ for red circles.
  • ax (matplotlib.axes.Axes or list thereof; optional, default: None) – A Matplotlib Axes object or a list thereof (one per electrode to plot). If None, a new Axes object will be created.
Returns:

axes (matplotlib.axes.Axes or np.ndarray of them) – Returns one matplotlib.axes.Axes per electrode

resize(shape, electrodes=None)[source]

Resize the video

Parameters:
  • shape ((rows, cols)) – Shape of each frame in the resized video. If one of the dimensions is set to -1, its value will be inferred by keeping a constant aspect ratio.
  • electrodes (int, string or list thereof; optional) –

    Optionally, you can provide your own electrode names. If none are given, electrode names will be numbered 0..N.

    Note

    The number of electrode names provided must match the number of pixels in the resized video.

Returns:

stim (VideoStimulus) – A copy of the stimulus object containing the resized video

rewind()[source]

Rewind the iterator

rgb2gray(electrodes=None)[source]

Convert the video to grayscale

Parameters:electrodes (int, string or list thereof; optional) –

Optionally, you can provide your own electrode names. If none are given, electrode names will be numbered 0..N.

Note

The number of electrode names provided must match the number of pixels in the grayscale video.

Returns:stim (VideoStimulus) – A copy of the stimulus object with all RGB values converted to grayscale in the range [0, 1].
rotate(angle, mode='constant')[source]

Rotate each frame of the video

Parameters:angle (float) – Angle by which to rotate each video frame (degrees). Positive: counter-clockwise, negative: clockwise
Returns:stim (VideoStimulus) – A copy of the stimulus object containing the rotated video
scale(scaling_factor)[source]

Scale the image foreground

This function scales the image foreground (excluding black pixels) by a factor.

Parameters:scaling_factor (float) – Factory by which to scale the image
Returns:stim (ImageStimulus) – A copy of the stimulus object containing the scaled image
shape

Data container shape

shift(shift_cols, shift_rows)[source]

Shift the image foreground

This function shifts the center of mass (CoM) of the image by the specified number of rows and columns.

Parameters:
  • shift_cols (float) – Number of columns by which to shift the CoM. Positive: to the right, negative: to the left
  • shift_rows (float) – Number of rows by which to shift the CoM. Positive: downward, negative: upward
Returns:

stim (ImageStimulus) – A copy of the stimulus object containing the shifted image

time

Time steps A list of time steps, corresponding to the columns in the data container.