Data Loss using data-stream Out in-db tool

[EnaK]

I have an issue with data stream out in-db tool and was wondering if anyone has a solution:

it is randomly losing data, example below shows same filters applied but different output as a result before and after data stream out used: 

[simonaubert_bd]

hello @EnaK you're supposed to have the same. Usually when it's not, it's a syntax issue in SQL or alteryx expression.

Can you do a screenshot of each filter and summarize configuration please ? In in-memory and in-db.

Best regards,

Simon

[EnaK]

adding screenshots and results now changed again... now it is ZERO vs 144

 

 

[apathetichell]

filter in-db requires the native sql syntax. most likely reason for this is that your sql syntax in your filters is not the same as your on-canvas filter syntax. the other thing I'd look at is if something went off on the type of numbers you are using (ie fixed witdth of 19.6 being converted to 25.0 (figures below .50 would be rounded to zero).

Also - turn off amp.

[EnaK]

I am sorry, I am only the beginner in this :) so, not sure I understand what you mean... I am not doing any conversion with the numbers anywhere in the process, why would it be different before and after?

also, how can I see the difference between SQL syntax and on-canvas filters?

thank you very much for your such quick responses!

[simonaubert_bd]

@EnaK Well, you must read the documentation of your database, each database having its own flavor of SQL (yeah, it pretty sucks).

Here, i guess you have issue on your filter because of the like operator that is not exactly similar to contains function.

Please have a look at https://www.w3schools.com/sql/sql_like.asp

Best regards,

Simon

[apathetichell]

One filter is 36444 - one is 364444. It has an extra 4. Fix. retry.

[EnaK]

i am sorry, that was just a typo to make up fake data, the filters in workflow are identical

[EnaK]

I might of stated my issue wrong.

the filter is not the issue, I only used it to find where the issue was occuring since I was randomly seeing different results of output.

once I'd get 120,000 records and running a minute later, I would get 10,000 then 130,000 and in reality, could actually be 160,000 records.

after testing and rerunning the data and checking results at different steps of the workflow I found the data was flowing all consistent until it hits the data stream out in-db tool... and that is when i decided to filter it down to smaller set of data to confirm the issue

so, PLEASE do not focus on filters :) thank you all again for all the help!

[apathetichell]

o.k. - so two things:

1) turn off amp.

2) what is your underlying database.

I assume it's not possible that the table you are querying is changing that rapidly?