VisTrails API Documentation¶

Module Definition¶

Module¶

class vistrails.core.modules.vistrails_module.Module¶

Module is the base module from which all module functionality is derived from in VisTrails. It defines a set of basic interfaces to deal with data input/output (through ports, as will be explained later), as well as a basic mechanism for dataflow based updates.

Execution Model

VisTrails assumes fundamentally that a pipeline is a dataflow. This means that pipeline cycles are disallowed, and that modules are supposed to be free of side-effects. This is obviously not possible in general, particularly for modules whose sole purpose is to interact with operating system resources. In these cases, designing a module is harder – the side effects should ideally not be exposed to the module interface. VisTrails provides some support for making this easier, as will be discussed later.

VisTrails caches intermediate results to increase efficiency in exploration. It does so by reusing pieces of pipelines in later executions.

Terminology

Module Interface: The module interface is the set of input and output ports a module exposes.

Designing New Modules

Designing new modules is essentially a matter of subclassing this module class and overriding the compute() method. There is a fully-documented example of this on the default package ‘pythonCalc’, available on the ‘packages/pythonCalc’ directory.

Caching

Caching affects the design of a new module. Most importantly, users have to account for compute() being called more than once. Even though compute() is only called once per individual execution, new connections might mean that previously uncomputed output must be made available.

Also, operating system side-effects must be carefully accounted for. Some operations are fundamentally side-effectful (creating OS output like uploading a file on the WWW or writing a file to a local hard drive). These modules should probably not be cached at all. VisTrails provides an easy way for modules to report that they should not be cached: simply subclass from the NotCacheable mixin provided in this python module. (NB: In order for the mixin to work appropriately, NotCacheable must appear BEFORE any other subclass in the class hierarchy declarations). These modules (and anything that depends on their results) will then never be reused.

Intermediate Files

Many modules communicate through intermediate files. VisTrails provides automatic filename and handle management to alleviate the burden of determining tricky things (e.g. longevity) of these files. Modules can request temporary file names through the file pool, currently accessible through self.interpreter.filePool.

The FilePool class is available in core/modules/module_utils.py - consult its documentation for usage. Notably, using the file pool will make temporary files work correctly with caching, and will make sure the temporaries are correctly removed.

_input_ports¶: Class attribute that stores the list of input ports for the module. May include instances of InputPort and CompoundInputPort.

_output_ports¶: Class attribute that defines the list of output ports for the module. May include instances of OutputPort and CompoundOutputPort.

_settings¶: Class attribute that stores a ModuleSettings object that controls appearance, configuration widgets, and other module settings.

compute()¶: This method should be overridden in order to perform the module’s computation.

get_input(port_name, allow_default=True)¶

Returns the value coming in on the input port named port_name.

Parameters:	port_name (str) – the name of the input port being queried allow_default (bool) – whether to return the default value if it exists
Returns:	the value being passed in on the input port
Raises:	`ModuleError` if there is no value on the port (and no default value if allow_default is True)

get_input_list(port_name)¶

Returns the value(s) coming in on the input port named port_name. When a port can accept more than one input, this method obtains all the values being passed in.

Parameters:	port_name (str) – the name of the input port being queried
Returns:	a list of all the values being passed in on the input port
Raises:	`ModuleError` if there is no value on the port

set_output(port_name, value)¶

This method is used to set a value on an output port.

Parameters:	port_name (str) – the name of the output port to be set value – the value to be assigned to the port

check_input(port_name) → None.¶

Raises an exception if the input port named port_name is not set.

Parameters:	port_name (str) – the name of the input port being checked
Raises:	`ModuleError` if there is no value on the port

has_input(port_name)¶

Returns a boolean indicating whether there is a value coming in on the input port named port_name.

Parameters:	port_name (str) – the name of the input port being queried
Return type:	bool

force_get_input(port_name, default_value=None)¶

Like get_input() except that if no value exists, it returns a user-specified default_value or None.

Parameters:	port_name (str) – the name of the input port being queried default_value – the default value to be used if there is no value on the input port
Returns:	the value being passed in on the input port or the default

force_get_input_list(port_name)¶

Like get_input_list() except that if no values exist, it returns an empty list

Parameters:	port_name (str) – the name of the input port being queried
Returns:	a list of all the values being passed in on the input port

annotate(d)¶

Manually add provenance information to the module’s execution trace. For example, a module that generates random numbers might add the seed that was used to initialize the generator.

Parameters:	d (dict) – a dictionary where both the keys and values are strings

ModuleError¶

class vistrails.core.modules.vistrails_module.ModuleError(module, errormsg, abort=False, errorTrace=None)¶: ModuleError should be passed the module instance that signaled the error and the error message as a string.

ModuleSettings¶

class vistrails.core.modules.config.ModuleSettings(name=None, configure_widget=None, constant_widget=None, constant_widgets=None, signature=None, constant_signature=None, color=None, fringe=None, left_fringe=None, right_fringe=None, abstract=False, package=None, namespace=None, version=None, package_version=None, hide_namespace=False, hide_descriptor=False, is_root=False, ghost_package=None, ghost_package_version=None, ghost_namespace=None)¶

name(String)¶: An optional string that will identify the module (defaults to the module class’s __name__).

configure_widget(QWidget | PathString)¶: An optional module configuration widget that is provided so that users can configure the module (e.g. PythonSource uses a widget to display a code editor more suited for writing code).

constant_widget(ConstantWidgetConfig | QWidget | PathString)¶: If not None, the registry will use the specified widget(s) or import path string(s) of the form “<module>:<class>” to import the widget. A tuple allows a user to specify the widget_type (e.g. “default”, “enum”, etc.) as the second item and the widget_use (e.g. “default”, “query”, “paramexp”). If the plural form is used, we expect a list, the singular form is used for a single item.

constant_widgets(List(constant_widget))¶: See ModuleSettings.constant_widget

signature(Callable)¶: If not None, then the cache uses this callable as the function to generate the signature for the module in the cache. The function should take three parameters: the pipeline (of type vistrails.core.vistrail.Pipeline), the module (of type vistrails.core.vistrail.Module), and a dict that stores parameter hashers. This dict is supposed to be passed to vistrails/core/cache/hasher.py:Hasher, in case that needs to be called.

constant_signature(Callable)¶: If not None, the cache uses this callable as the function used to generate the signature for parameters of the associated module’s type. This function should take a single argument (of type vistrials.core.vistrail.module_param.ModuleParam) and returns a SHA hash.

color((Float, Float, Float))¶: In the GUI, the module will be colored according to the specified RGB tuple where each value is a float in [0,1.0].

fringe(List((Float, Float)))¶: If not None, generates custom lateral fringes for the module boxes. module_fringe must be a list of pairs of floats. The first point must be (0.0, 0.0), and the last must be (0.0, 1.0). This will be used to generate custom lateral fringes for module boxes. All x values must be positive, and all y values must be between 0.0 and 1.0. Alternatively, the user can set ModuleSettings.left_fringe and ModuleSettings.right_fringe to set two different fringes.

left_fringe(List((Float, Float)))¶: See ModuleSettings.fringe.

right_fringe(List((Float, Float)))¶: See ModuleSettings.fringe.

abstract(Boolean)¶: If True, means the module only serves as a base class for other modules. It will not appear in the module palette but may be used as a input or output port type.

package(String)¶: If not None, then we use this package instead of the current one. This is only intended to be used with local per-module module registries (in other words: if you don’t know what a local per-module registry is, you can ignore this option).

namespace(String)¶: If not None, then we associate a namespace with the module. A namespace is essentially appended to the package identifier so that multiple modules inside the same package can share the same name. Namespaces may be nested using the “|” separator. For example, “Tools|Mathematical” specified the Mathematical namespace inside of the Tools namespace.

version(String)¶: The module version. This is usually used with subworkflows where the underlying module may have different versions. Not recommended for general use at this time.

package_version(String)¶: The current package version for the module. As with ModuleSettings.package, you shuold not use this unless you know what you are doing as VisTrails will automatically fill in this information for any normal package.

hide_namespace(Boolean)¶: If True, the module palette will not display the namespace for the module. Used for subworkflows, otherwise it may be confusing for users.

hide_descriptor(Boolean)¶: If True, the module palette will not display that module in its list (similar to abstract, but can be used when the module is not truly abstract).

is_root(Boolean)¶: Internal use only. This is used to designate the base Module class and should not be used by any other module.

ghost_package(String)¶: If not None, then the ‘ghost_identifier’ is set on the descriptor, which will cause the module to be displayed under that package in the module palette, rather than the package specified by the package argument (or current package).

ghost_package_version(String)¶: If not None, then the attribute ‘ghost_package_version’ is set on the descriptor. Currently this value is unused, but eventually if multiple packages with the same identifier but different package versions are loaded simultaneously, this will allow overriding the package_version to clean up the module palette.

ghost_namespace(String)¶: If not None, the descriptor will be displayed under the specified namespace instead of the ‘namespace’ attribute of the descriptor.

Port Specification¶

InputPort (IPort)¶

class vistrails.core.modules.config.InputPort(name=None, signature=None, optional=False, sort_key=-1, docstring=None, shape=None, min_conns=0, max_conns=-1, depth=0, label=None, default=None, values=None, entry_type=None)¶

name(String)¶: The name of the of the port

signature(String)¶: The signature of the of the port (e.g. “basic:Float”)

optional(Boolean)¶: Whether the port should be visible by default (defaults to True)

sort_key(Integer)¶: An integer value that indicates where, relative to other ports, this port should appear visually

docstring(String)¶: Documentation for the port

shape( "triangle" | "diamond" | "circle" | [(Float,Float)])¶: The shape of the port. If triangle, appending an angle (in degrees) rotates the triangle. If a list of (x,y) tuples, specifies points of a polygon in the [0,1] x [0,1] region

min_conns(Integer)¶: The minimum number of values required for the port

max_conns(Integer)¶: The maximum number of values allowed for the port

depth(Integer)¶: The list depth of the port. Default is 0 (no list)

label(String)¶: A label to be shown with a port

default¶: The default value for a constant-typed port

values(List)¶: A list of enumerated values that a ConstantWidgetConfig uses to configure the widget. For example, the “enum” widget uses the entries to present an exclusive list of choices.

entry_type(String)¶: The type of the configuration widget that should be used with this port. Developers may use custom widgets on a port-by-port basis by adding widgets with different ConstantWidgetConfig.widget_type values and defining the port’s entry_type to match.

class vistrails.core.modules.config.IPort¶: Synonym for InputPort

OutputPort (OPort)¶

class vistrails.core.modules.config.OutputPort(name=None, signature=None, optional=False, sort_key=-1, docstring=None, shape=None, min_conns=0, max_conns=-1, depth=0)¶

name(String)¶: The name of the of the port

signature(String)¶: The signature of the of the port (e.g. “basic:Float”)

optional(Boolean)¶: Whether the port should be visible by default (defaults to True)

sort_key(Integer)¶: An integer value that indicates where, relative to other ports, this port should appear visually

docstring(String)¶: Documentation for the port

shape( "triangle" | "diamond" | "circle" | [(Float,Float)])¶: The shape of the port. If triangle, appending an angle (in degrees) rotates the triangle. If a list of (x,y) tuples, specifies points of a polygon in the [0,1] x [0,1] region

min_conns(Integer)¶: The minimum number of values required for the port

max_conns(Integer)¶: The maximum number of values allowed for the port

depth(Integer)¶: The list depth of the port. Default is 0 (no list)

class vistrails.core.modules.config.OPort¶: Synonym for OutputPort

CompoundInputPort (CIPort)¶

class vistrails.core.modules.config.CompoundInputPort(name=None, signature=None, optional=False, sort_key=-1, docstring=None, shape=None, min_conns=0, max_conns=-1, depth=0, items=None, labels=None, defaults=None, values=None, entry_types=None)¶

name(String)¶: The name of the of the port

signature(String)¶: The signature of the port (e.g. “basic:Integer, basic:Float”). Note that for compound ports, this may be instead included on a per-component basis in InputPortItem.signature.

optional(Boolean)¶: Whether the port should be visible by default (defaults to True)

sort_key(Integer)¶: An integer value that indicates where, relative to other ports, this port should appear visually

docstring(String)¶: Documentation for the port

shape( "triangle" | "diamond" | "circle" | [(Float,Float)])¶: The shape of the port. If triangle, appending an angle (in degrees) rotates the triangle. If a list of (x,y) tuples, specifies points of a polygon in the [0,1] x [0,1] region

min_conns(Integer)¶: The minimum number of values required for the port

max_conns(Integer)¶: The maximum number of values allowed for the port

depth(Integer)¶: The list depth of the port. Default is 0 (no list)

items(List(InputPortItem))¶: Either use this field and break individual labels/defaults/values/entry_types into InputPortItem components or use the other four fields.

labels(List(String))¶: A list of InputPort.label

defaults(List)¶: A list of InputPort.default

values(List(List))¶: A list of InputPort.values

entry_types(List(String))¶: A list of InputPort.entry_type

class vistrails.core.modules.config.CIPort¶: Synonym for CompoundInputPort

CompoundOutputPort (COPort)¶

class vistrails.core.modules.config.CompoundOutputPort(name=None, signature=None, optional=False, sort_key=-1, docstring=None, shape=None, min_conns=0, max_conns=-1, depth=0, items=None)¶

name(String)¶: The name of the of the port

signature(String)¶: The signature of the port (e.g. “basic:Integer, basic:Float”). Note that for compound ports, this may be instead included on a per-component basis in OutputPortItem.signature.

optional(Boolean)¶: Whether the port should be visible by default (defaults to True)

sort_key(Integer)¶: An integer value that indicates where, relative to other ports, this port should appear visually

docstring(String)¶: Documentation for the port

shape( "triangle" | "diamond" | "circle" | [(Float,Float)])¶: The shape of the port. If triangle, appending an angle (in degrees) rotates the triangle. If a list of (x,y) tuples, specifies points of a polygon in the [0,1] x [0,1] region

min_conns(Integer)¶: The minimum number of values required for the port

max_conns(Integer)¶: The maximum number of values allowed for the port

depth(Integer)¶: The list depth of the port. Default is 0 (no list)

items(List(OutputPortItem))¶: Either use this field and break individual signatures into OutputPortItem components or use the signature field.

class vistrails.core.modules.config.COPort¶: Synonym for CompundOutputPort

InputPortItem (IPItem)¶

class vistrails.core.modules.config.InputPortItem(signature=None, label=None, default=None, values=None, entry_type=None)¶

signature(String)¶: See InputPort.signature

label(String)¶: See InputPort.label

default¶: See InputPort.default

values(List)¶: See InputPort.values

entry_type(String)¶: See InputPort.entry_type

class vistrails.core.modules.config.IPItem¶: Synonym for InputPortItem

OutputPortItem (OPItem)¶

class vistrails.core.modules.config.OutputPortItem(signature=None)¶

signature(String)¶

See InputPort.signature

class vistrails.core.modules.config.OPItem¶: Synonym for OutputPortItem

Parameter Widget Configuration¶

ConstantWidgetConfig¶

class vistrails.core.modules.config.ConstantWidgetConfig(widget, widget_type=None, widget_use=None)¶

widget(QWidget | String)¶: The widget to be used either as the class itself or a string of the form <py_module>:<class> (e.g. “vistrails.gui.modules.constant_configuration:BooleanWidget”)

widget_type(String)¶: A developer-available type that links a widget to a port entry_type

widget_use(String)¶: Intended to differentiate widgets for different purposes in VisTrails. Currently uses the values None, “query”, and “paramexp” to define widget uses in parameter, query, and parameter exploration configurations, repsectively.

QueryWidgetConfig¶

class vistrails.core.modules.config.QueryWidgetConfig(widget=None, widget_type=None, widget_use='query')¶

widget(QWidget | String)¶: The widget to be used either as the class itself or a string of the form <py_module>:<class> (e.g. “vistrails.gui.modules.constant_configuration:BooleanWidget”)

widget_type(String)¶: A developer-available type that links a widget to a port entry_type

widget_use(String)¶: Like ConstantWidgetConfig.widget_use, just defaulted to “query”

ParamExpWidgetConfig¶

class vistrails.core.modules.config.ParamExpWidgetConfig(widget=None, widget_type=None, widget_use='paramexp')¶

widget(QWidget | String)¶: The widget to be used either as the class itself or a string of the form <py_module>:<class> (e.g. “vistrails.gui.modules.constant_configuration:BooleanWidget”)

widget_type(String)¶: A developer-available type that links a widget to a port entry_type

widget_use(String)¶: Like ConstantWidgetConfig.widget_use, just defaulted to “paramexp”