Parsing Inputs

Compression: The selected compression method decompresses the input data files.

• Select the required compression method from the dropdown.

For Example:

GZip: GNU Zip

Encryption: The selected encryption method decrypts the data using an algorithm and private key to parse the information.

For Example:

PGP: PGP encryption selection will enable the Configure button.

None: This selection will keep disabling the Configure button, which will not allow you for algorithm selections.

Configure: Encryption algorithms assist in the process of transforming original data into encrypted data using the public key.

Configuration

1. Click the Configure button. The Configuration Pop-up will appear.
2. Select the required encryption method from the Select Method dropdown.
3. Click Upload Private Key to upload the file or paste the private key in the text box.
4. Enter the Passphrase in the text box.
5. Click on the View icon to view and confirm the entered Passphrase.
6. Click Save Changes to save the encryption configuration.

The configured Passphrase will not be visible while editing the configuration. The uploaded key will be used to decrypt the encrypted data.

For Example: 

PGP: Pretty Good Privacy (PGP), which works through cryptography, data compression, and hashing techniques.

None: Select None if data files are not encrypted.

Directory: The parent directory helps to define the file's static path. Also, it automatically reflects at the bottom of the File Name Pattern (Regex) with the regular expression.

The Final Path combines the parent directory and regular expression corresponding to the file/directory in S3.

Amazon S3 bucket must have read and write access, so you can use following functionality as mentioned below:

Automatic Folder Structure:

• The batch listener automatically creates a folder structure in the provided path.
• This structure is maintained to track processed files and facilitate troubleshooting in case of failures.

File Handling:

• When the listener completes reading a file, it is moved to a "Processed" folder within the same directory.
• This relocation is done to prevent reprocessing.

Reprocessing Capability:

• If you need to reprocess the same file, simply move it from the "Processed" directory back to the original path.
• The listener will pick up the file for processing once again.

Always use one directory per listener, separated by "/."

For Example: Index/plans/^L…*C$

If the parent directory is empty, then the final path will be considered as "Root".

For Example: {root}/^L…*C$

Path Pattern (Regex): Regular expressions are mainly helpful for defining filters. It contains a series of characters that define a pattern of text to be matched to make a filter more specific.

Available data files in the source location can have different detail patterns. You can provide regular expressions to cover all of those different patterns. There can be several combinations with respect to names, like fixed directory-fixed file names, fixed directory name-dynamic file names, and dynamic directory name-dynamic file names.

For Example:

CSV files are placed in a directory named "data" that needs to be parsed.

For more information about Regular Expressions, See Path Pattern (Regex).

Related Topics

• Batch Listener
• Path Pattern (Regex)
• Data Format
• Interval 
• Output