File Read Adaptor

Creating a File Read Adaptor

The File Read Adaptor reads the contents of an object-based file.

• To create a new item via the Create menu item in the project editor, select the File Adaptor icon, and give the adaptor a name. 
• Choose the connection from the drop down menu, and choose the file name and file contents destination.

File Name: The File Name value can be provided by a literal, schema value, or public variables

• For S3, this name should include any path outside of the top-level bucket.

File Contents Destination: The location for the body of the read file, and can be a schema value (recommended) or a public variable.

If the file contains a single JSON object, this will be parsed and expanded at the location specified. If the file contains more than one JSON object or non-JSON data, then the complete contents of the file will be saved as a string field at the specified location. Be wary of loading large files in this method, as it is designed to load small objects and not large files. 

Currently, only Amazon Web Services Simple Storage System (AWS S3) is supported, but it is planned to expand to other types (for example, Azure blob storage).

• AWS S3 is an object-based storage scheme presenting a file-system view of the objects it stores; folders are termed buckets, and files are objects. For all intents and purposes, it appears like a web-hosted file system. The ability to read and write data from S3 buckets helps integrate with other systems that can only handle file-based integrations, such as DMPs. 
• There is currently no way to list all of the files in a bucket - but this is something that we plan to add. A sample use case may be to have a stream of file names that require to be processed and the file reader processing those files. 

Validation Warnings

: All the invalid notifications will be displayed after expanding the Invalid Drop-down.

Related Pages:

• File Write Adaptor