Workflow Malfunction

So I have made a custom standard macro as per my need. That macro has a python tool that fetches data from the Cloud. When I call that macro into the workflow and run the workflow, it runs properly. The issue arises when I run two workflows with the same macro simultaneously.

Very unusual behavior occurs. Let's say I'm fetching table1 in workflow number 1 using my standard macro, and table 2 in workflow number 2 using the same macro. What happens is that workflow 2 throws an error which is as follows:

RuntimeError: Error creating the file " C:\Users\AppData\Local\Temp\Engine_3348_8e44062fd7924c478191425d277d1d82_\0328e956-f7d3-44fa-af6d-262d3ce5e43c\output_1.yxdb ": The process cannot access the file because it is being used by another process. (32)

and workflow 1 outputs the data of workflow 2.

What could be the issue?

Workflow

Python

Macros

Accepted answers

grazitti_sapna

@mihir_sampat , just a small query have you tried to run workflow in parallel by using two different datasets?

I mean for workflow 1 you can use the dataset you are referring and for workflow 2 try using maybe a copy of dataset 1 but with different name. Usually what happens is if you are trying to access same file but different tabs this issue might occur so I would like you to test this scenario that I mentioned then we can try to investigate further.

Thanks.

All comments

grazitti_sapna

HI @mihir_sampat , this might be an issue for running workflows at the same time which is trying to access the file while the workflow is still in a running condition, try using block until done tool after your workflow 1 and workflow 2 input.

Another thing you can try is using dynamic input tool since you are using the same file to fetch different tables, unless the schema is same and all the transformations are same you can do it in a single workflow as well. If the scheme is different and the transformations differ then unfortunately you have to give the above solution a try.

I hope this helps.

Thanks.

mihir_sampat

Actually, I have to make the workflow run parallel to each other where I will be getting the data from different tables from the Google Cloud and then load them to the cloud at the same time.

Quick Links

This months top contributors

mceleavey 383

mbarone 337

Hollingsworth 335

LanisC 335

JeffF 335