Transforming Data
An important facility in DataView is the ability to transform data in various ways. You can never alter data in an original acquisition file: all transformed data have to be written into a file in the native DataView dtvw-dat format. But data in dtvw-dat format files can be manipulated in almost any way you like.
Data transformations are all accessed through the Transform menu. If you hover over a menu command, you will see a brief summary of the command function at the left-hand end of the status bar at the bottom of the main frame window.
This section gives a simple example to illustrate the general features of data transformation. More details of specific transformations are given in the Data Transformation and Analysis sections of the tutorial.
E.g. De-Mean, Rectify, Smooth
A common stage in the analysis of extracellular recordings is to change the "spiky" waveform of the raw recording into a smoothly-changing waveform whose moment-by-moment amplitude reflects the intensity of the spike activity. This may be done in real-time with a hardware integrator built into the recording amplifier itself, but it can also be done post-hoc within DataView.
- Load the file tadpole rhythm.
This shows a fragment of extracellular recording from a spinal motor root during an episode of swimming in a tadpole. We will transform the spike bursts into a series of smooth hills to make subsequent analysis of the rhythm easier. - Select the Transform: De-mean/median, rectify, smooth menu command to activate the Multi-process transform dialog box.
- Check each of the De-mean/median data, Rectify about 0 and Smooth (moving average) boxes. The dialog box should look like the image below:
These settings mean that the data will first be de-meaned (have the overall mean value subtracted from all samples – this is only needed if there is an offset to the extracellular recording baseline), then full-wave rectified, and finally smoothed with 1 iteration of a moving-average filter with a half-window of 1 ms.
- There is only one data trace in the recording, but if there were more than one trace, we could apply these transformations to any number of traces by entering a list of desired trace ID numbers (separated by spaces, or hyphens for sequences) into the Traces to proces edit box.The order of entry does not matter.
- By default the mean is calculated from the whole file, but if desired the mean could be calculated from only a selected section of the file. This might be useful if the file was very big (it would be quicker), or if part of the file was known to contain only “background” activity which would give a truer representation of the mean than the whole file.
Preview transformations
Most transforms allow you to preview the results of the transform before applying them. This enables you to fine-tune transform parameters before committing to the actual transform.
- Click the Preview button.
A new window will display showing two traces. The upper trace is the raw data from the first trace listed in the Traces to process box of the controlling transform dialog. The lower trace (shown in red) is the raw data transformed with the current parameters selected in the transform dialog. You can switch between this preview and the controlling dialog at will, and if you change parameters in the controlling dialog, the preview is automatically updated.
- In the transform dialog set the Half window Time to 4 ms (or the Samples to 20; the Time and Samples edit options are interchangeable).
- Increase the Iterations to 4, and observe the progressive changes in the Preview display.
- In the preview dialog click the Autoscale toolbar button (
: the same as in the main view). The preview display should now like this.
A single iteration of a moving average filter is equivalent to convolving the data with a rectangular kernel (hence the alternative names “boxcar” (a type of American railway carriage) or “tophat” (a type of British hat) smoothing), while performing 4 or more iterations of the moving average approximate to convolving with a Gaussian kernel, hence the increased smoothness.
The preview window can only display a single trace: to change the trace, edit the the controlling dialog to make the desired trace the first in the list. You can also change the display gain settings, and navigate through the source file so as to preview the transformation applied to different sections of data using the preview toolbar buttons, which have equivalent functionality to those in the main window. However, you cannot preview transformations containing more than 2 million data points, so you cannot preview an entire transformation of a very large file.
Superimpose Raw and Transformed Data
- Click the Superimpose checkbox
This superimposes the raw and transformed data within a single axis, which can help see the relative timing of the two traces. It can also help when filtering data to see the difference between the raw and transformed data. Note that when superimposed, the gain and position of the two traces cannot be manipulated independently, since these are set by the axis on which the traces display.
Transform Destination
Three options determined how the transformed data are treated:
- A Write new file checkbox is available on all transform dialogs. If you select this you will be asked to specify a new file to which the data will be written. This option is automatically selected and locked for original acquisition files, so you are forced to write to a new dtvw-dat file.
- The Replace/Add trace(s) choice determines whether the transformed data replaces the original data trace (Replace option) or whether a new trace is written to contain the transformed data (Add option). In the latter case the new trace will be inserted so as to be next in sequence after the original data trace.
- The Batch process checkbox allows you to apply the same transformation to multiple files, without having to explicitly call up the transform dialog box for each file. This is explained in more detail elsewhere.
To activate the transformation and actually make it happen, click the OK button. If you want to abandon the transformation without changing the data, click Cancel. Both OK and Cancel dismiss the transform dialog.
Parameter Persistence
The Save button enable you to save the parameters that you have entered into the dialog box to a binary configuration file (with the extension dtvw-cfg), and the Load button loads parameters from a saved configuration file. Note that the configuration files for different transformations have different formats, so you can only load a configuration file into a dialog if it has been previously saved from the same dialog. You will be alerted (hopefully without a program crash) if you attempt to load an inappropriate configuration file.
Note: clicking Save does not activate the transformation itself, it just saves the configuration.
You can also load a configuration file and open its dialog directly from the File: Load config file menu command.
Drag-and-Drop
A configuration file can be loaded and its dialogs opened by dragging from File Explorer and dropping it onto the main view.
You can also drop a configuration file onto an appropriate dialog which is already open, which is the equivalent to using the Load facility within the dialog.
History: Keeping track of transforms
The transformation is performed when you click the OK button. A brief summary of the transformation is appended to the trace label of the transformed data, to help you keep track of what you have done. You can, of course, edit this label as desired using the Traces: Format menu command.
The full parameter list of the transform is appended to the History text accessed through the File: History facility.
You can also enter notes into the Append comment box. The text in this box is appended to the main file comment, which can be viewed through the File: Info command, and edited through the File: Comment command.