labtools API¶

Stream¶

class otestpoint.labtools.Stream(endpoint, *probes, **kwargs)[source]¶

Factory for streaming measurements to variables.

Creates a Stream instance.

Keyword Arguments:
Parameters:	endpoint (str) – OpenTestPoint probe publisher in the form host:port. probes (str) – One or more OpenTestPoint probe names.
	cache (int) – Number of measurements to cache per reporting tag. Default: 120. notify_threshold (int) – Number of milliseconds to delay variable notification after receiving a measurement. Default: 500. rate (int) – Rate in seconds the OpenTestPoint probes are set to publish. Default: 5.
Raises:	`KeyError` – If an invalid keyword is found.

Example:

s = Stream('localhost:9002',
           'EMANE.PhysicalLayer.Counters.General',
           'EMANE.TDMAEventSchedulerRadioModel.Counters.Schedule',
           'EMANE.TDMAEventSchedulerRadioModel.Counters.General',
           'EMANE.TDMAEventSchedulerRadioModel.Tables.Status.Slot',
           'EMANE.VirtualTransport.Tables.Status')

cache()[source]¶

Gets the stream cache size

Returns:	Cache size.
Return type:	int

model(*subjects, **kwargs)[source]¶

Creates a data model.

Keyword Arguments:
Parameters:	subjects (object) – Tuple of variables (with or without modifiers) used to populate the model.
	by_tag (bool) – Indicate if the model should using tags as the index or timestamps. Default: False. labels ([str]) – Names of the model columns. If empty, column default names are made using measurement and attribute names. Default: [].
Returns:	New model instance.
Return type:	`Model`
Raises:	`KeyError` – If invalid keyword found.

Example:

model = stream.model(Delta(Transform(varRxSlotStatusTable,
                             TableColumns(3))),
             Delta(Transform(varRxSlotStatusTable,
                             TableColumns(4,5,6,7,8))),
             Delta(Transform(varTxSlotStatusTable,
                             TableColumns(3))),
             Delta(Transform(varTxSlotStatusTable,
                             TableColumns(4,5))),
             labels=['Rx Slot Success',
                     'Rx Slot Error',
                     'Tx Slot Success',
                     'Tx Slot Error'],
             by_tag=True)

run()[source]¶: Starts the processing for OpenTestPoint probe measurements.

stop()[source]¶: Stops the processing for OpenTestPoint probe measurements.

variable(measurement, attribute, **kwargs)[source]¶

Creates a Variable instance that will receive attribute measurement updates.

You must be sure to instantiate the Stream with a probe that provides the specified measurement in order to receive updates.

Keyword Arguments:
Parameters:	measurement (str) – Name of the measurement to associate with the variable. attribute (str) – Name of the attribute contained in the measurement to associate with the variable.
	apply (callable) – Callable applied to a measurement prior to storing. Useful to reduce storage to just the relevant measurement information of interest. Default: lambda x: x. cache (int) – Number of measurements to cache per reporting tag. Default: Value set for stream. tags ([str]) – Names of tags to include measurements from. If empty all tags are included. Default: [].
Returns:	New variable instance.
Return type:	`Variable`
Raises:	`KeyError` – If an invalid keyword is found.

Example:

v = stream.variable('Measurement_emane_physicallayer_counters_general',
    'processedevents')

Variable¶

class otestpoint.labtools.variable.Variable(stream, measurement, attribute, **kwargs)[source]¶

Creates a Variable instance that will receive attribute measurement updates.

You must be sure to instantiate the Stream with a probe that provides the specified measurement in order to receive updates.

Keyword Arguments:
Parameters:	measurement (str) – Name of the measurement to associate with the variable. attribute (str) – Name of the attribute contained in the measurment to associate with the variable.
	apply (callable) – Callable applied to a measurment prior to storing. Useful to reduce storage to just the relevant measurement information of interest. Default: lambda x: x. cache (int) – Number of measurments to cache per reporting tag. Default: Value set for stream. tags ([str]) – Names of tags to include measurments from. If empty all tags are included. Default: [].
Raises:	`KeyError` – If an invalid keyword is found.

name()[source]¶

Gets the variable name

Returns:	Variable name.
Return type:	str

Model¶

class otestpoint.labtools.model.Model(stream, *subjects, **kwargs)[source]¶

Creates a data model.

Keyword Arguments:
Parameters:	stream (obj) – Stream associated with the model. subjects (Subject) – Tuple of variables (with or without used to populate the model. (modifiers)) –
	by_tag (bool) – Indicate if the model should using tags as the index or timestamps. Default: False. labels ([str]) – Names of the model columns. If empty, column default names are made using measurement and attribute names. Default: []. purge (bool) – Indicate if the model should purge values that are no longer cached by subjects. Default: True.
Raises:	`KeyError` – If an invalid keyword is found.

Example:

model = stream.model(Delta(Transform(varRxSlotStatusTable,
                             TableColumns(3))),
             Delta(Transform(varRxSlotStatusTable,
                             TableColumns(4,5,6,7,8))),
             Delta(Transform(varTxSlotStatusTable,
                             TableColumns(3))),
             Delta(Transform(varTxSlotStatusTable,
                             TableColumns(4,5))),
             labels=['Rx Slot Success',
                     'Rx Slot Error',
                     'Tx Slot Success',
                     'Tx Slot Error'],
             by_tag=True)

by_tag()[source]¶

Indicates if the model is using tags as the index or timestamps

Returns:	True for indexed by tag, False otherwise.
Return type:	bool

data(**kwargs)[source]¶

Gets a dataframe of current measurements.

Keyword Arguments:
	index (int) – Numerical index of column to use as the dataframe index or None for timestamps. Default None. join (str) – Type of join to use for non-timeseries dataframes. One of: ‘left’,’right’,’outer’ or ‘inner.’ Default: ‘outer.’ apply (str) – Callable applied to dataframe prior to creating or None. Default: None. ts (bool) – Indicates if dataframe is a timeseries. Default: True.
Returns:	`(dataframe,event count, (timestamp min, timestamp max))` Where the event count (int) indicates the index (count) of the current model that was used to generate the dataframe. The event count increases with every received measurement. This allows for determining if the resulting dataframe is potentially different from the previous dataframe. The timestamp min (int) and timestamp max (int) values indicate the current measurement window in seconds since the epoch.
Return type:	tuple
Raises:	`KeyError` – If an invalid keyword is found. `ValueError` – If a keyword value is unallowed. `ModelError` – If a timeseries dataframe is requested from a model not organized by timestamp.

Example:

import time
import pandas as pd

event_prev = 0

pd.set_option('display.max_columns', None)
pd.set_option('display.expand_frame_repr', False)

while True:
    df,event_cur,(_,timestamp) = model.data(ts=False,
                                            index=None)

    if event_cur != event_prev:
        print df

        event_prev = event_cur

    time.sleep(1)

stream()[source]¶

Gets the stream associated with the model.

Returns:	`Stream`

Delta Modifier¶

class otestpoint.labtools.Delta(variable, **kwargs)[source]¶

Produce delta values from subject data.

Delta modifiers work on subjects with single value state, list of value state and list of list of value state:

val

[val_0,val_1,…val_N]

[[val_0,val_1,…val_N],…,[val_0,val_1,…val_N]]

Creates a Delta modifier instance.

Keyword Arguments:
Parameters:	variable (Subject) – Variable to modify.
	name (str) – Modified variable name. Default: Delta(variable).

Example:

model = Model(Sum(Delta(varSchedOK1),
                  Delta(varSchedOK2)),
              Sum(Delta(varSchedErr1),
                  Delta(varSchedErr2),
                  Delta(varSchedErr3),
                  Delta(varSchedErr4)),
              ...)

name()[source]¶

Gets the name of this observer.

Returns:	Modified variable name.
Return type:	str

notify(subject)[source]¶: Processes subject update to produce new state.

state()[source]¶

Gets the state.

Returns:	A dict of with timestamp (int) keys and dict values. Where each dict value has tag name (str) keys and the delta value. Depending on the subject, that can be a single value, a list of values or a list of list of values.
Return type:	dict

Sum Modifier¶

class otestpoint.labtools.Sum(*subjects, **kwargs)[source]¶

Produces summation value of subject data.

Creates a Sum instance.

Sum instances work on subjects with single value state or list of values state:

val

[val_0,val_1,…val_N]

Parameters:	subjects ([subject]) – One or more variables to sum.

Keywords Args:

name (str): Modified variable name. Default:: Sum(variable[+variable]…).

Raises:	`KeyError`

Example:

model = Model(Sum(Delta(varSchedOK1),
                  Delta(varSchedOK2)),
              Sum(Delta(varSchedErr1),
                  Delta(varSchedErr2),
                  Delta(varSchedErr3),
                  Delta(varSchedErr4)),
              ...)

name()[source]¶

Gets the name of this subject.

Returns:	Subject name (str).

notify(subject)[source]¶: Processes subject update to produce new state.

state()[source]¶

Gets the state.

Returns:	A dict of with timestamp (int) keys and dict values. Where each dict value is the summation value.

Transform Modifier¶

class otestpoint.labtools.Transform(variable, transform, **kwargs)[source]¶

Transform subject data using a callable.

Creates a Transform instance.

Transform instances work on subjects with single value state or list of values state:

val [val_0,val_1,…val_N]

Parameters:	variable (str) – transform (callable) –

Kwargs:

name (str): Subject name. Default: Transform(transform(variable)).

Raises:	`KeyError`

name()[source]¶

Gets the name of this subject.

Returns:	Subject name (str).

notify(subject)[source]¶: Processes subject update to produce new state.

state()[source]¶

Gets the state.

Returns:	A dict of with timestamp (int) keys and dict values. Where each dict value has tag name (str) keys and the transformed value. Depending on the subject, that can be a single value, a list of values.

Unique Modifier¶

class otestpoint.labtools.Unique(subject, **kwargs)[source]¶

Produces count of unique values subject data.

Creates a Unique instance.

Unique instances work on subjects with single value state:

val

Keyword Arguments:
Parameters:	subject (Subject) – Variable to modify.
	name (str) – Modified variable name. Default: Unique(variable).

Example:

model = ModelTimeSeries(Unique(varSomeId)),
                        ...)

name()[source]¶

Gets the name of this subject.

Returns:	Subject name (str).

notify(subject)[source]¶: Processes subject update to produce new state.

state()[source]¶

Gets the state.

Returns:	A dict of with timestamp (int) keys and dict values. Where each dict value is the unique count.

Subject¶

class otestpoint.labtools.subject.Subject(stream)[source]¶

Observer¶

class otestpoint.labtools.observer.Observer[source]¶

View¶

class otestpoint.labtools.View(model, kind, **kwargs)[source]¶

Line.Plot¶

class Line.Plot(*columns, **kwargs)[source]¶

Creates a Line Plot.

A line plot is a strip chart of one or more time-series measurments.

Keyword Arguments:
Parameters:	columns ([str]) – One or more DataFrame column names to plot.
	title (str) – Plot title. Default: ‘’. ylim ((min,max)) – Plot y-axis limit plot limit. Default: (0,50000). ylabel (str) – Plot y-axis label. Default: ‘’. markers ([str]) – One or more DataFrame columns to use as markers. A marker is a vertical line that indicates an event occurred. A marker is drawn any time the indicated DataFrame column value is non-zero and non-nan. Default: []. indicators ([str]) – One or more DataFrame columns to use as indicators. An indicator is a horizontal bar that indicates a condition is met. An indicator is continuously drawn for time stamps where the indicated DataFrame column value is non-zero and non-nan. Default: []. legend (bool) – Flag indicating whether a plot legend should be displayed. Default: True.
Raises:	`KeyError` – If an invalid keyword is found.

Bar.Plot¶

class Bar.Plot(*columns, **kwargs)[source]¶

Creates a Bar Plot.

A bar plot is a plot of the latest timestamp data for one or more measurements.

Keyword Arguments:
Parameters:	columns ([str]) – One or more DataFrame column names to plot.
	title (str) – Plot title. Default: ‘’. yticks ([int]) – Plot y-axis tick. Default: None. ylabel (str) – Plot y-axis label. Default: ‘’. xlabel (str) – Plot x-axis label. Default: ‘’. xtick_rotation (int) – Percentage of rotation to apply to xtick labels. Default: 70. legend (bool) – Flag indicating whether a plot legend should be displayed. Default: False.
Raises:	`KeyError` – If an invalid keyword is found.

Heatmap.Plot¶

class Heatmap.Plot(*columns, **kwargs)[source]¶

Creates a Heatmap Plot.

A heatmap plot is a colorbar chart of a time-series measurment.

Keyword Arguments:
Parameters:	columns ([str]) – One or more DataFrame column names to plot.
	title (str) – Plot title. Default: ‘’. ylim ((min,max)) – Plot y-axis limit plot limit. Default: None. markers ([str]) – One or more DataFrame columns to use as markers. A marker is a vertical line that indicates an event occurred. A marker is drawn any time the indicated DataFrame column value is non-zero and non-nan. Default: []. indicators ([str]) – One or more DataFrame columns to use as indicators. An indicator is a horizontal bar that indicates a condition is met. An indicator is continuously drawn for time stamps where the indicated DataFrame column value is non-zero and non-nan. Default: []. legend (bool) – Flag indicating whether a plot legend should be displayed. Default: False.
Raises:	`KeyError` – If an invalid keyword is found.

DataFrameBuilder¶

class otestpoint.labtools.DataFrameBuilder(files, *probes, **kwargs)[source]¶

Builds Pandas DataFrames from variables defined for recorded measurment attributes.

Creates a DataFrameBuilder instance.

Parameters:	files (list(str)) – List of otestpoint-recorder output data files. The corresponding SQLite otestpoint-recorder db files must match data file name + ‘.db’. probes (str) – One or more OpenTestPoint probe names.
Raises:	`KeyError` – If an invalid keyword is found.

Example:

df_builder = DataFrameBuilder(['node-01-otestpoint-recorder.data',
                               'node-02-otestpoint-recorder.data',
                               'node-03-otestpoint-recorder.data'],
             'EMANE.PhysicalLayer.Counters.General',
             'EMANE.TDMAEventSchedulerRadioModel.Counters.Schedule',
             'EMANE.TDMAEventSchedulerRadioModel.Counters.General',
             'EMANE.TDMAEventSchedulerRadioModel.Tables.Status.Slot',
             'EMANE.VirtualTransport.Tables.Status')

load(**kwargs)[source]¶

Loads the data model and builds one or more dataframes from recorded measurments.

Keyword Arguments:
	index (int) – Numerical index of column to use as the dataframe index or None for timestamps. Default None. join (str) – Type of join to use for non-timeseries dataframes. One of: ‘left’,’right’,’outer’ or ‘inner.’ Default: ‘outer.’ apply (str) – Callable applied to dataframe prior to creating or None. Default: None. ts (bool) – Indicates if dataframe is a timeseries. Default: True.
Returns:	`tuple(dataframe, timestamp min, timestamp max)` Where the timestamp min (int) and timestamp max (int) values indicate the current measurement window in seconds since the epoch.
Return type:	list
Raises:	`KeyError` – If invalid keyword found. `ImportError` – If unable to import a probe measurement.

Example::

dfs = df_builder.load(apply=cbwu)

if dfs:: df,timestamp_start,timestamp_end = dfs[0] df.to_csv(‘dataframe.csv’);

model(*subjects, **kwargs)[source]¶

Creates the data model using transforms and variables.

Keyword Arguments:
Parameters:	subjects (object) – Tuple of variables (with or without modifiers) used to populate the model.
	by_tag (bool) – Indicate if the model should using tags as the index or timestamps. Default: False. labels ([str]) – Names of the model columns. If empty, column default names are made using measurement and attribute names. Default: [].
Raises:	`KeyError` – If invalid keyword found.

Example:

model = df_builder.model(Delta(Transform(varRxSlotStatusTable,
                             TableColumns(3))),
             Delta(Transform(varRxSlotStatusTable,
                             TableColumns(4,5,6,7,8))),
             Delta(Transform(varTxSlotStatusTable,
                             TableColumns(3))),
             Delta(Transform(varTxSlotStatusTable,
                             TableColumns(4,5))),
             labels=['Rx Slot Success',
                     'Rx Slot Error',
                     'Tx Slot Success',
                     'Tx Slot Error'],
             by_tag=True)

variable(measurement, attribute, **kwargs)[source]¶

Creates a Variable instance that will receive attribute measurement updates.

You must be sure to instantiate the DataFrameBuilder with a probe that provides the specified measurement in order to receive updates.

Keyword Arguments:
Parameters:	measurement (str) – Name of the measurement to associate with the variable. attribute (str) – Name of the attribute contained in the measurement to associate with the variable.
	apply (callable) – Callable applied to a measurement prior to storing. Useful to reduce storage to just the relevant measurement information of interest. Default: lambda x: x. tags ([str]) – Names of tags to include measurements from. If empty all tags are included. Default: [].
Returns:	New variable instance.
Return type:	`Variable`
Raises:	`KeyError` – If an invalid keyword is found.

Example:

v = df_builder.variable('Measurement_emane_physicallayer_counters_general',
    'processedevents')

labtools API¶

Stream¶

Variable¶

Model¶

Delta Modifier¶

Sum Modifier¶

Transform Modifier¶

Unique Modifier¶

Subject¶

Observer¶

View¶

Line.Plot¶

Bar.Plot¶

Heatmap.Plot¶

DataFrameBuilder¶

Table Of Contents

Previous topic

This Page