Working with Files

Use the following blocks to work with files:

• Generate CSV
• Archive
• Unpack
• Extract text from PDF
• Upload to the catalog
• Get file list
• Get CSV
• Download the contents of the file
• Delete file

When using blocks from the Data storage category, each workspace automatically gets its own dedicated folder named as a GUID (unique identifier). This folder is created in the Operavix working directory under the filestorage subfolder:

• On Windows: %ProgramData%\Operavix
• On Linux (inside the container): /var/lib/operavix/data/

Generate CSV

This block creates a CSV file from the data received from the previous blocks.

Parameters:

• CSV columns:
  • Column name
  • Column data (added via visual mapping)
• Add column headings (if enabled, the first row of headings becomes column names)
• Separator:
  • Semicolon
  • Comma
  • Pipeline
  • Slash
  • Tab
  • Space
  • Other (specify the separator in the field that appears)

The generated file can be archived and uploaded to a directory.

Archive

The block compresses file data received from a previous block into the gzip format.

Specify the file to be archived in the File content field via visual mapping.

The resulting archive can be uploaded to a directory and unpacked.

Unpack

The block extracts the contents of a gzip file.

Specify the archive to be unpacked in the File content field via visual mapping. You can also specify the name and extension for the unpacked file.

Extract Text from PDF

This block extracts text from a PDF file.

In the block parameters, use mapping to provide the PDF file content in the File field.

The extracted text can then be used in subsequent blocks.

Upload to the Catalog

The block creates a file and uploads it to a specified directory in any format.

Parameters of the block:

• Path to file:
  • For the Data storage package blocks, specify the path relative to the filestorage directory of your workspace
  • For the FTP, SFTP, and SMB package blocks, specify the path relative to the workspace root set in the connection settings
  • The path must include the filename and extension (for example, /folder_name/file_name.csv)
  • For archives, use the .gzip extension
  • You can also provide the path via mapping
• File content (only mapping fields are supported). It may contain:
  • Text
  • Text with mapping tags (content must be uniform: cannot mix text and images)
  • Mapping tags only (also must be uniform)

Get File List

This block retrieves a list of files, their properties, and directory information.

In the Directory path field:

• For Storage package blocks, specify the path relative to the filestorage directory of your workspace
• For FTP, SFTP, and SMB package blocks, specify the path relative to the workspace root set in the connection settings
• You can enter the path manually or via mapping

The block returns an array of data:

• Name of the file/directory (String)
• Size of the file/directory in bytes (String)
• Owner of the file/directory (String)
• Group of the file/directory owner (String)
• Modification_time — the time of the last modification of the file/directory (String)
• Permissions of the user to the file/directory, shown as a sequence of characters indicating access rights (read, write, execute) for the owner, group, and other users. A dash indicates no access

Get CSV

The block retrieves a CSV file from a specified directory.

Parameters:

• Path to file:
  • For the Data storage package blocks, specify the path relative to the filestorage directory of your workspace
  • For the FTP, SFTP, and SMB package blocks, specify the path relative to the workspace root set in the connection settings
  • The path must include the filename and extension
• The table contains a header (specifies whether the CSV contains a header row)
• Column names (can be reordered by dragging):
  • Column name
  • Data type (String, Integer, Integer Number, Date, Date and time, Boolean, BigInteger, BigDecimal)
• Upload example (allows uploading a CSV file to auto-add columns):
  • Data types are detected automatically but can be edited
  • Uploading a new schema replaces the previous one
  • Option to save parsing settings to block parameters
• Parsing settings:
  • Separator:
  • Semicolon
  • Comma
  • Pipeline
  • Slash
  • Tab
  • Space
  • Other (custom input)
  • Qualifier:
  • Single quotes
  • Double quotes
  • Locale (applied during parsing)
  • Encoding (applied when processing the file’s byte stream)

Uploading an Example

For columns to be added automatically, upload a sample CSV file.

To do it, click Upload example to the right of the Column names header. A window will open for uploading a file.

Select a file and click Continue.

A preview window opens where you can change the separator, qualifier, locale, encoding, and column data types. If you select the Save parsing settings option, the settings will be copied to the parameters of the block. To finish the process click Apply.

The columns will be automatically added, replacing the previous ones.

Download the Contents of the File

The block allows you to download a file from the specified directory.

• For the Data storage package blocks, specify the path relative to the filestorage directory of your workspace
• For the FTP, SFTP, and SMB package blocks, specify the path relative to the workspace root set in the connection settings
• The path must include the filename and extension (for example, /folder_name/file_name.csv)
• The path can be provided manually or via mapping

Files downloaded with this block are stored in the buffer and can be reused by other file-handling blocks without re-fetching from storage.

Delete File

The block deletes a file from the specified directory.

• For the Data storage package blocks, specify the path relative to the filestorage directory of your workspace
• For the FTP, SFTP, and SMB package blocks, specify the path relative to the workspace root set in the connection settings
• The path must include the filename and extension (for example, /folder_name/file_name.csv)
• The path can be provided manually or via mapping