Download (1): Error transferring data: Error in the SSH layer

[preethisisaac]

Hi All,

Am able to download contents of the sftp file sftp://sftp.sense.com/outgoing/20200902_webvisit.csv

separately with download tool.

Now i tried listing the root folders with download tool and got the below:

sftp://sftp.sense.com/ drwxr--r-- 1 - - 0 June 1 2020 incoming
sftp://sftp.sense.com/ drwxr--r-- 1 - - 0 June 1 2020 outgoing

But when i tried to list down the files in sftp://sftp.sense.com/outgoing/

Am getting below error, after the workflow running for 30mins.

Error: Download (1): Error transferring data: Error in the SSH layer

What might be the reason why am not able to get the list of .csv files.

Am able to see these files with winscp also.

Please suggest options.

Thanks,

Preethi

[messi007 · Accepted solution]

Dear @preethisisaac,

Have you tried to use a batch macro in roder to read file by file not all at the same time:

Below the macro:

Below the workflow that will call the macro in order to read file by file

Attached the workflow. Hope that helps,

Regards,

[preethisisaac]

Hi @messi007 ,

IF i give a list of .csv files in text input tool, i am able to download and read the files one after another and write the whole to output tool.

But my requirement is to take as input the start date and the end date with interface date tool , and generate dates between the range and form the file names and then use download tool and read the files.I am able to do it. But if any of the generated files name does not exist in the sftp folder, I am getting an error " file not found" and the workflow quits with the error.

So i have  to now compare the list of filenames i have generated and the actual file names in the sftp folder and confirm if available as TRUE and if not available as FALSE, and then filter it for TRUE and then read the file contents.

continue...

It is for this purpose, i am trying to pull the list of files from the sftp folder.

I have about 1350 files in the sftp folder which keeps on growing as days pass by.

Is this a huge list for the download tool to list out.

Please suggest how to pull in the list of .csv files from the sftp directory.

Why am i getting the Error: Download (1): Error transferring data: Error in the SSH layer

Thanks,

[bertal34]

@messi007 

Your batch macro solution was very helpful for me.  I did have to change the Action tool connected to the Download to update the Header field @name value instead of the setting it was currently on.  Thank you!

[messi007]

@bertal34 I'm happy that my solution helped!