when importing a folder with multiple files it automatically concats the files which is perfect but gives me wrong data set because files are having different number of columns, different column names, only two columns (in all files) are matching, rest is different.
My goal is to union all the files. I have managed this by importing all the files manually from the S3 folder and then do Union as shown in the below image:
Currently, there are two files but in future it will be more. How can I automate importing all the files which are in specific folder and then do the UNION and UNPIVOTING automatically.