Select and Basic Data Profile Not Showing Correct Counts

[stapuff]

I have a query Select * that pulls all fields/records.

207 Fields - 496,446 records both confirmed.

SELECT TOOL - 

I have a Field MANDT that has '500' in every record. The profile in the select tool should show 496,446 but it only shows 172,876

BASIC DATA PROFILE TOOL - 

I have the same query connected with the same results.

496,446 records

OK - 172,876

NULLS - 0

NON-NULLS - 172,876

BLANKS - 0

The 172,876 appears regardless of if all values in the field are the same or different.

Any thoughts on where 172,876 is coming from or how to "fix" it to show correct counts?

Thanks,

Puff

Select ToolBasic Data Profile Tool

[pedrodrfaria]

Hi Stapuff, the issue could be that the Browse Tool (what you called the Select Tool) has a limit at 300MB for its size, so it won't load "everything everything" if your dataset is super large (Help Page). What is the size of the dataset you are currently working with? It could also be a bug for the newest release.

[stapuff · Accepted solution]

@pedrodrfaria 

You are correct... it is a Browse Tool. 108MB total size.  Edit: just watched the dataset rerun.... 827MB.

<500,000 records and half the fields are blank.

The exact reason I wanted to use the Basic Data Profile tool... Identify and remove the useless fields.

Everything was working great until I stepped to a bigger data set.

Puff

[aahaseebkhan]

I've a similar error, i'm parsing 458,241 records but the profile tool only processes 353,280. 

BASIC DATA PROFILE TOOL - 

458,241 records

OK - 353,280

NULLS - 0

NON-NULLS - 353,280

BLANKS - 0

Initially i've tried it with over 200M records and it only processed 200K records. I'm not using any select tool here, the input is directly connected with basic data profile tool.