Adding FTP Input to a Flow

With the ftp input node, you can use an FTP upload as a trigger for actions in your flow. While there are many ways you could use this functionality, one common use is to run TDE templates automatically when an input file is uploaded via FTP.

Using this node requires the following setup:
 

•Create Secure FTP (SFTP) user accounts.
•Assign at least one SFTP user to the flow.
Multiple SFTP users can be assigned to a single flow, but each SFTP user can only be assigned to one flow at a time.
•Add an ftp input node to the flow to watch for FTP uploads from the SFTP user accounts assigned to the flow.
•Set up an SFTP connection using the SFTP client of your choice.
To find the SFTP Hostname and Port required for this connection, click the down arrow to the right of the flow and select "Edit Secure FTP users".

Assigning SFTP Users to a Flow

 

1.From the Developer Console, click the down arrow to the right of the flow you want to edit, and select "Edit Secure FTP users".
2.From the Unassigned Secure FTP users drop-down menu, select the SFTP user to assign.
3.Click Assign to this flow.
Any FTP uploads from this user’s credentials are now watched automatically by any ftp input nodes added to the flow.

Sample Flow: Using FTP Input to Execute a TDE Template

After setting up SFTP users, you can add the ftp input node to your flows.

The example below shows how you can use the ftp input node to watch for an upload of a TDE input file, parse the CSV contents of the file into a JSON object, and then automatically run a TDE template with that data.

Keep in mind that the configuration options described below are setup to handle the test data provided in this sample. When implementing a similar flow, you likely need to adjust these options to meet the needs of your particular use case.

You must have TDE authentication credentials to execute this flow.

 

1.Create the sample flow:
a.From the Developer Console, create a new flow named sftp.
b.Click the down arrow to the right of the flow, and select "Edit Secure FTP users".
c.Assign a Secure FTP user to the flow.
Make sure you know the credentials for this user, so that you can upload the input file used later in this example.
2.Click the flow name (sftp) to open it in the Flow Editor.
3.From the Flow Editor, add and connect the following nodes:
a.From the storage section of the nodes palette, add the ftp input node.
b.From the parser section, add the csv node to the right of the ftp input node.
c.Connect the output of the ftp input node to the input of the csv node.
d.From the telogis section, add the tde template node to the right of the csv node.
e.Connect the csv node output to the tde template node input.
f.From the utility section, add the debug node to the right of the tde template node.
g.Connect the tde template node output to the debug node input.
4.Configure the ftp input node:
a.Double-click the ftp input node to open the edit screen.
b.Select the option to "Use a regular expression" for the filename.
c.For Match filename, enter CreateDriver.*txt.
This regular expression can be read as follows: 1) CreateDriver matches the characters CreateDriver literally (case sensitive), 2) .* matches any character, between zero and unlimited times, and 3) txt matches the characters txt literally (case sensitive). For example, CreateDriver-20160505.txt and CreateDriver1.txt both match.
d.For Output, select "Contents".
e.For Datatype, select "String".
f.Click Done.
5.Configure the csv node:
a.Double-click the csv node to open the edit screen.
b.For the Separator, select "comma".
c.For the CSV to object options:
i.For the Input, select the option "first row contains column names".
ii.For the Output, select "a message per row".
iii.For Parse, select the option "convert numbers to javascript numbers".
d.For the Object to CSV options:
i.For the Output, select the option "include column name row".
ii.For Newline, select "unix (\n)".
e.Click Done.
The CSV file that you upload by FTP in a later step is parsed by the csv node and converted into a JSON object that is passed to the tde template node.
6.Configure the tde template node:
a.Double-click the tde template node to open the edit screen.
b.For the Config, select an existing TDE configuration or add a new one as needed.
c.Copy and paste the following template into the template area:
 

TemplateVersion = 1.0

TableID = Driver-1.0

TemplateName = CreateDriver2

Intent = Create

Commit = false

Format = json

 

[Output]

Id(BasicDriver) = Input.EmployeeNumber

Nickname = Input.FirstName || " " || Input.LastName

Surname = Input.LastName

FirstName = Input.FirstName

 

d.Select the option to "Execute template with data".
By selecting this option, the data in the JSON object passed from the csv node is used when the template is executed.
e.For the msg. property below Execute template with data, enter payload.
f.Click Done.
Note that this sample template has set Commit = false, so that this data is only validated and not committed.
7.Configure the debug node:
a.Double-click the debug node to open the edit screen.
b.For the Output, select "complete msg object".
c.Click Done.
The debug node is only added for testing purposes, so that you can see the results of an FTP upload in the Flow Editor.
8.Deploy your flow.
9.Upload an input file by FTP:
a.Using the FTP client of your choice, establish a connection to the flow. You need the following information:
•Depending on how you connect, either the SFTP user’s username and password or the user’s SSH key.
•The SFTP Hostname and Port. You can find this information from the Developer Console. Click the down arrow to the right of the flow and select "Edit Secure FTP users".
b.Create and save an input file locally named CreateDriver1.txt with the following content:
 

FirstName,LastName,EmployeeNumber

Dina,Deluna,33445

Dale,Donnell,33456

 

c.Upload CreateDriver1.txt to the flow using your FTP client.
10.View the results of the upload:
a.After a few seconds, status messages are displayed under the nodes on the central sheet, indicating that an upload has been detected and the data is moving through the flow.
b.From the right information panel, click the debug tab.
The tab should display an array containing tde status information showing that the request was completed.

Source Code

The flow created in this tutorial is described by the JSON below. To successfully execute the flow, you must edit the tde template node Config to add your own TDE credentials before you deploy.

To import a flow into the Editor, copy the code provided, click the upper right ≡ Menu button, then select "Import" > "Clipboard". Paste the code in the Import nodes window, then click Import.

Code:
 

[

    {

        "id": "ab76f2dd.2c47d",

        "type": "debug",

        "z": "f9095e7d.e308d",

        "name": "",

        "active": true,

        "console": "false",

        "complete": "true",

        "x": 650,

        "y": 100,

        "wires": []

    },

    {

        "id": "ff9fae96.20f9a",

        "type": "ftp-watcher",

        "z": "f9095e7d.e308d",

        "filename": "CreateDriver.*txt",

        "useRegex": true,

        "output": "contents",

        "datatype": "utf8",

        "x": 80,

        "y": 100,

        "wires": [

            [

                "c4e0a922.9042"

            ]

        ]

    },

    {

        "id": "2fd834e.d26becc",

        "type": "tde template",

        "z": "f9095e7d.e308d",

        "name": "",

        "template": "[Template]\nTemplateVersion = 1.0\nTableID = Driver-1.0\nTemplateName = CreateDriver2\nIntent = Create\nCommit = False\nFormat = json\n\n[Output]\nId(BasicDriver) = Input.EmployeeNumber\nNickname = Input.FirstName || \" \" || Input.LastName\nSurname = Input.LastName\nFirstName = Input.FirstName\n",

        "executeTemplateWithData": true,

        "templateDataLocation": "payload",

        "useMustacheTemplating": false,

        "mustacheDataLocation": "__undefined__",

        "templateMessageData": "",

        "mustacheMessageData": "",

        "outputDataLocation": "payload",

        "useStreaming": false,

        "config": "2cb5c1c2.26cc4e",

        "x": 470,

        "y": 100,

        "wires": [

            [

                "ab76f2dd.2c47d"

            ]

        ]

    },

    {

        "id": "c4e0a922.9042",

        "type": "csv",

        "z": "f9095e7d.e308d",

        "name": "",

        "sep": ",",

        "firstRowContainsColumnNames": true,

        "parse": true,

        "outputColumnNameRow": true,

        "sendMultipleMessages": "true",

        "ret": "unix",

        "x": 270,

        "y": 100,

        "wires": [

            [

                "2fd834e.d26becc"

            ]

        ]

    },

    {

        "id": "2cb5c1c2.26cc4e",

        "type": "api-config",

        "z": "f9095e7d.e308d",

        "proxy": "",

        "instance": "production",

        "custominstance": "",

        "labelValue": "production - techdev14:main",

        "useMessageCredentials": "false",

        "credentialsCustomer": "techdev14",

        "credentialsUser": "main"

    }

]