# File-Based I/O in Python

The user-defined functions feature (UDF) gives you the ability to perform file-based IO. This lets you apply a function to long memory depths.

For deep waveforms, you can instead use a file to pass the data in and out of a UDF. To use file-based IO, the Python script needs to read from and write to a file.

In most cases, you will want to process a part of the source data at a time so as to not create internal Python arrays that are too large.

Also, the .XML file must contain a keyword to tell the oscilloscope to use file IO. This line should be added to the .XML file that configures the oscilloscope:

    <FileIO>true</FileIO>

Instead of passing SrcData variables to Python, the oscilloscope passes SrcDataFile variables. These variables are strings that are the file name and path to the files that contain source data.

In the Python script, create a variable 'FiltDataFile' that is the path and file name of the resultant data.

The source and function data files are binary files of type 'float'.

Here is an example XML configuration file for a file-based I/O user-defined function:

RealScale.xml
Drag the mouse over this listing, enter Ctrl-C to copy, and paste it into the XML text editor.
    <?xml version="1.0" encoding="utf-8"?>
    <Function xmlns:xsd="http://www.w3.org/2001/XMLSchema"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       Application="Infiniium" MinimumSoftwareVersion="22.1">
       <Name>RealScale</Name>
       <Abbreviation>Scale</Abbreviation>
       <Comments>Scales the input waveform.</Comments>
       <Script>RealScale.py</Script>
       <Icon></Icon>
       <FileIO>true</FileIO>
       <FunctionType>1 Source</FunctionType>
       <Control Name = "ScaleFactor">
          <Double>
             <Min>-100</Min>
             <Max>100</Max>
             <Resolution>1e-3</Resolution>
             <Units>None</Units>
             <Default>3</Default>
          </Double>
       </Control>
    </Function>

Here is an example script that uses file-based I/O:

RealScale.py
Drag the mouse over this listing, enter Ctrl-C to copy, and paste it into the Python editor.
    import numpy as np
    import os
    import struct

    def algorithm(variables):

        # Ensure ScaleFactor is defined.
        ScaleFactor = variables['ScaleFactor'] if 'ScaleFactor' in variables else 3.0

        # Set the output file location to the same directory as the input but with a new name.
        SrcDataFile = variables['SrcDataFile'] if 'SrcDataFile' in variables else "test_src_data.bin"
        filepath, filename = os.path.split(SrcDataFile)
        name, ext = os.path.splitext(filename)
        FiltDataFile = os.path.join(filepath, f'{name}_scaled{ext}')

        # Define the chunk size (100,000 bytes).
        chunk_size = 100000

        # Open the respective files for reading/writing.
        with open(SrcDataFile, 'rb') as src_file, open(FiltDataFile, 'wb') as filt_file:
            while True:
                file_bytes = src_file.read(chunk_size)
                if not file_bytes:
                    break

                # Convert bytes to list of floats (4 bytes per float).
                num_floats = len(file_bytes) // 4
                floats = list(struct.unpack(f'{num_floats}f', file_bytes[:num_floats * 4]))

                # Apply scaling.
                scaled_floats = [f * ScaleFactor for f in floats]

                # Write back as 4-byte floats.
                filt_file.write(struct.pack(f'{len(scaled_floats)}f', *scaled_floats))

        # Inputs copied to the outputs.
        XOrg = variables['XOrg'] if 'XOrg' in variables else 0.0
        XInc = variables['XInc'] if 'XInc' in variables else 0.0

        # Other variables customized.
        Gain = 2.0
        FilterWidth = 0.0
        FilterDelay = 0.0
        XUnits = 'Same'
        YUnits = 'Same'
        ErrorMsg = ''

        # Output these variables.
        output_variables = {
            "FiltDataFile" : FiltDataFile,
            "XOrg" : XOrg,
            "XInc" : XInc,
            "Gain" : Gain,
            "FilterWidth" : FilterWidth,
            "FilterDelay" : FilterDelay,
            "XUnits" : XUnits,
            "YUnits" : YUnits,
            "ErrorMsg" : ErrorMsg
        }
        return output_variables

    # When testing from the command line.
    if __name__ == "__main__":
        print(algorithm({'SrcDataFile': 'test_src_data.bin', 'XOrg': 0.1, 'XInc': 1.1}))