topic Re: gzip tool unable to read file meta data, when gml is gzipped inside in Alteryx Designer Desktop Discussions

gzip tool unable to read file meta data, when gml is gzipped inside

steven4320555 — Sun, 24 Jun 2018 22:44:05 GMT

The anticipated gzip support was recently released in 2018.2, I finally got some time to get started with some of OS Mastermap data stored as gz with gml inside.

Unfortunately, I got the following message when I tried to open the gz file.

"error extracting file: unable to read file meta data"

I guess it might have something to do with the fact that gz read only support limited file format, although gml is supported if it is not gzipped. (as I shared in https://community.alteryx.com/t5/Alteryx-Designer-Ideas/Support-for-GZIP-gz-compressed-files-in-input-tool-both-in/idc-p/176758#M6819 )

Just want to bring this issue to the attention of the developers and the community, in case someone might have a similar problem, or may have time and may be able to provide some signpost.

One workaround I can think of is to unzip the gz file into temporary files (I have over a thousand, would not be a problem for batch macro), and then use the existing gml capactiy. But as a lazy user, who is a little scared and short of time to use the run command tool, as introduced here: https://tarsolutions.co.uk/blog/unzip-a-file-in-alteryx/ . Just to check if supporting gml will be soon released in beta version, I am happy to help test it out.

Many thanks

Steven

Re: gzip tool unable to read file meta data, when gml is gzipped inside

blyons — Mon, 25 Jun 2018 18:58:47 GMT

I don't think it is restricted to whether you have gml inside. I am getting the same error with a standard .GZipped text file. See https://community.alteryx.com/t5/Alteryx-Designer-Discussions/GZIP-support-not-working-in-2018-2-Input-tool/m-p/177159#M45917

Re: gzip tool unable to read file meta data, when gml is gzipped inside

DanM — Mon, 25 Jun 2018 19:49:44 GMT

@blyons and @steven4320555,

The gzip files must be a tar.gz and at this point we only support the file formats below.

The following file formats can be read from a .tar.gz file and extracted to a temporary location on your hard drive:

.cydb, .dbf, .kml, .grc, .grd, .qvx, .sas7bdat, .sav, .sz, .xlsm, .xlsx, .xlsx (legacy), .xls, .xml

https://help.alteryx.com/current/DataSources/GZipFiles.htm <https://help.alteryx.com/current/DataSources/GZipFiles.htm>

DanM

Re: gzip tool unable to read file meta data, when gml is gzipped inside

blyons — Mon, 25 Jun 2018 20:04:19 GMT

That is extremely disappointing, especially since that was not what anyone in the idea thread even requested.

Re: gzip tool unable to read file meta data, when gml is gzipped inside

steven4320555 — Tue, 26 Jun 2018 11:13:00 GMT

Thanks for the reply. Is there any plan for include more file type in the near future? Will it be something in the beta scheme if you may know?

Re: gzip tool unable to read file meta data, when gml is gzipped inside

steven4320555 — Tue, 26 Jun 2018 11:14:39 GMT

People will need to do it step by step, but it good to take users need into consideration, especially some big users. I am a free academic account user, so I am not asking more.

Re: gzip tool unable to read file meta data, when gml is gzipped inside

ARich — Tue, 26 Jun 2018 22:53:12 GMT

Hi @steven4320555,

We will look at adding gml as a supported file inside a gzipped archive. Gml is not one of our most frequently used file types, so we didn't include it in the first round, but we can certainly iterate and add! Thanks for the feedback!

Best,

Alex

Re: gzip tool unable to read file meta data, when gml is gzipped inside

ARich — Tue, 26 Jun 2018 23:11:39 GMT

Hi @blyons,

Thanks for the feedback. Since .gz doesn't provide any metadata, we ran into some technical issues implementing straight .gz files. Therefore, we touched base with some users and decided on .tar.gz extensions as a first iteration. We're looking into what's necessary for all .gz files at the moment. Apologies for the disappointment and we hope to get this to you soon!

Just to confirm- your use case is reading in a log file from a .gz archive, correct? What is the the file type of your log file inside the archive?

Best,

Alex

Re: gzip tool unable to read file meta data, when gml is gzipped inside

steven4320555 — Wed, 27 Jun 2018 09:19:21 GMT

@ARich Thanks very much for the quick reply. It is good news. Just to confirm if the plan will be to add gml support in .gz? Or it has to be .tar.gz?

My user case is some bulk downloaded data from OS Mastermap topographical layer in GB. I used FME to handle the data, but it took days, I am looking forward to see what Alteryx can do. As you might know recently, there is some good move towards opening up more open spatial data in UK, and a lot of the spatial data is in .gml format (it might be an incentive for Alteryx to prioritize the support). https://www.agi.org.uk/news/agi/1297-open-mastermap-webinar-recording-available ; https://www.gov.uk/government/news/unlocking-of-governments-mapping-and-location-data-to-boost-economy-by-130m-a-year ; https://www.ordnancesurvey.co.uk/about/news/2018/mastermap-announcement.html

Another user case, for gml data in UK and EU will be the INSPIRE polygon (they are not gzipped and can be opened with Alteryx directly). eg. in England and Wales "The INSPIRE Index Polygons can help you locate registered freehold land and property in England and Wales; the data is updated monthly." https://www.gov.uk/guidance/inspire-index-polygons-spatial-data#about-inspire-polygons

GML format

GML is an open data format that is widely used across Europe and complies with the EU INSPIRE Directive. By supplying data in GML format, HM Land Registry support the use of open standards and supply a standard data structure to users of all GIS/CAD systems.

Re: gzip tool unable to read file meta data, when gml is gzipped inside

blyons — Wed, 27 Jun 2018 15:05:31 GMT

My files are text inside .gz. A sample can be found attached to this thread: https://community.alteryx.com/t5/Alteryx-Designer-Discussions/GZIP-support-not-working-in-2018-2-Input-tool/m-p/177159#M45917

Re: gzip tool unable to read file meta data, when gml is gzipped inside

ARich — Wed, 27 Jun 2018 17:26:23 GMT

@steven4320555 and @blyons,

Thank you very much for the specifics and info. The plan is as follows and will be done in the following order: 1. add .gml format to existing .zip and .tar.gz functionality. We're hoping to get this first part in very soon. 2. Add .gz support, which will support the same formats that already exist in .zip and .tar.gz archives currently (this will include gml since it will be part of step 1). Text files will also be included in this, you'll just need to tell us how to read the data in. We are also hoping to get this in fairly soon and will make it available for beta prior to release to ensure we're meeting the majority of needs.

Again, thanks for your patience and feedback.

Best,

Alex

Re: gzip tool unable to read file meta data, when gml is gzipped inside

lisander — Tue, 16 Oct 2018 09:16:32 GMT

Hi there,

In my case I am outputting a blob field from a SQL DB, which is a .gz. In the file I have a .xml

Outputting as a file is easy, but as I am currently dealing with almost 2 million files and each an xml with almost 7000 headers, would be great to sample/filter fields before writing it into another DB.

But anyway, beiing able to extract .gz would be great for first :-)