Reading counts of 100 files

[Pradeep66]

Hi team, 

I know I can read 100 files using dynamic input through directory tool. Is there a way to find out all list of files and it's record count with out opening the files? 

It's like directory tool gives size of files without opening the files, similarly can we get no of records for each file? 

Because my files are very bigger and opening just for sake of fetching getting file counts might not be a good idea. 

Appreciate your ideas. 

[IraWatt]

Hey @Pradeep66,

Will be interesting to see what other people think but I dont think it is possible to get a row count without opening the flat file first as for instance, CSV's/XML files dont have that as information as metadata. What file types are you looking at?

[Felipe_Ribeir0]

Hi @Pradeep66 ! I agree with @IraWatt , you cannot know this without importing the whole file. 

What you could do is, if you have access to the system that is the source of these files and the system allow you to do it,  put this information on the name of the file. Like filename_200345.csv, for a file with 200345 rows. Or even only one .txt file with this info for each file inside of it.

[Felipe_Ribeir0]

One alternative if you consider to use python, probably a lot faster than importing it with the input tool:

https://stackoverflow.com/questions/845058/how-to-get-line-count-of-a-large-file-cheaply-in-python

[Pradeep66]

my files are csv and excels.

[Pradeep66]

this is nice thought to append the record count to file name for future quick access. Saves reading record times.

[Pradeep66]

still trying to figure out a way which can be reusable for any place of workflow.