Reading counts of 100 files

Hi team,

I know I can read 100 files using dynamic input through directory tool. Is there a way to find out all list of files and it's record count with out opening the files?

It's like directory tool gives size of files without opening the files, similarly can we get no of records for each file?

Because my files are very bigger and opening just for sake of fetching getting file counts might not be a good idea.

Appreciate your ideas.

Custom Tools

Accepted answers

All comments

IraWatt

Hey @Pradeep66,

Will be interesting to see what other people think but I dont think it is possible to get a row count without opening the flat file first as for instance, CSV's/XML files dont have that as information as metadata. What file types are you looking at?

Felipe_Ribeir0

Hi @Pradeep66 ! I agree with @IraWatt , you cannot know this without importing the whole file.

What you could do is, if you have access to the system that is the source of these files and the system allow you to do it, put this information on the name of the file. Like filename_200345.csv, for a file with 200345 rows. Or even only one .txt file with this info for each file inside of it.

Felipe_Ribeir0

One alternative if you consider to use python, probably a lot faster than importing it with the input tool:

https://stackoverflow.com/questions/845058/how-to-get-line-count-of-a-large-file-cheaply-in-python

Pradeep66

my files are csv and excels.

Pradeep66

this is nice thought to append the record count to file name for future quick access. Saves reading record times.

Pradeep66

still trying to figure out a way which can be reusable for any place of workflow.

Quick Links

This months top contributors

atcodedog05 19598

Qiu 15880

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703