Community Spring Cleaning week is here! Join your fellow Maveryx in digging through your old posts and marking comments on them as solved. Learn more here!

Dev Space

Customize and extend the power of Alteryx with SDKs, APIs, custom tools, and more.

Install (compile) spacy with pip python sdk

Nikolay
6 - Meteoroid

Hi

I’m facing a problem, it’s not exactly Alteryx problem but still I cannot use Alteryx because of it. I’m currently working on text mining projects and I want to use spacy. I tried to install it before with pip in windows but it cannot compile it (I tried nearly all versions of visual studio build tools without any success) so I moved to ubuntu. But now I want to create an Alteryx workflow with python sdk. How are you installing python libraries, which need to be compiled first in Windows ?

11 REPLIES 11
PaulN
Alteryx Alumni (Retired)

Hi @Nikolay,

 

Are you able to run "pip install spacy" from the Scripts folder of your tool repository ?

 

 

 

C:\Program Files\Alteryx\bin\Miniconda3>python -m venv %appdata%\Alteryx\Tools\SpacyTest

cd %appdata%\Alteryx\Tools\SpacyTest

cd Scripts

pip install spacy

 

In my case, installation works but it generates warnings related to numpy.

Example:

 

C:\Program Files\Alteryx\bin\Miniconda3\lib\importlib\_bootstrap.py:205: RuntimeWarning: numpy.dtype size changed, may i
ndicate binary incompatibility. Expected 96, got 88
return f(*args, **kwds)

 

 

Note that you will have to install models through pip (or requirements.txt), example: 

 

 

pip install en_core_web_sm

 

 

Thanks,

 

 

Paul Noirel

Sr Customer Support Engineer, Alteryx

 

Nikolay
6 - Meteoroid

Hi @

Thank you for the fast reply. I installed nltk with pip. But when I run pip install spacy I'm getting:

Installing collected packages: numpy, murmurhash, cymem, preshed, msgpack, msgpack-numpy, toolz, cytoolz, wrapt, plac, tqdm, pyreadline, dill, thinc, ujson, regex, idna, chardet, urllib3, certifi, requests, spacy
  Running setup.py install for murmurhash ... error
    Complete output from command c:\users\nikolay\appdata\local\alteryx\bin\htmlplugins\pythonsdktest\scripts\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\NIKOLA~1\\AppData\\Local\\Temp\\pip-build-znq14ndh\\murmurhash\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\NIKOLA~1\AppData\Local\Temp\pip-1rqp36vy-record\install-record.txt --single-version-externally-managed --compile --install-headers c:\users\nikolay\appdata\local\alteryx\bin\htmlplugins\pythonsdktest\include\site\python3.6\murmurhash:
    running install
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-3.6
    creating build\lib.win-amd64-3.6\murmurhash
    copying murmurhash\about.py -> build\lib.win-amd64-3.6\murmurhash
    copying murmurhash\__init__.py -> build\lib.win-amd64-3.6\murmurhash
    creating build\lib.win-amd64-3.6\murmurhash\tests
    copying murmurhash\tests\test_import.py -> build\lib.win-amd64-3.6\murmurhash\tests
    copying murmurhash\tests\__init__.py -> build\lib.win-amd64-3.6\murmurhash\tests
    copying murmurhash\mrmr.pyx -> build\lib.win-amd64-3.6\murmurhash
    copying murmurhash\mrmr.pxd -> build\lib.win-amd64-3.6\murmurhash
    copying murmurhash\__init__.pxd -> build\lib.win-amd64-3.6\murmurhash
    creating build\lib.win-amd64-3.6\murmurhash\include
    creating build\lib.win-amd64-3.6\murmurhash\include\murmurhash
    copying murmurhash\include\murmurhash\MurmurHash2.h -> build\lib.win-amd64-3.6\murmurhash\include\murmurhash
    copying murmurhash\include\murmurhash\MurmurHash3.h -> build\lib.win-amd64-3.6\murmurhash\include\murmurhash
    running build_ext
    building 'murmurhash.mrmr' extension
    creating build\temp.win-amd64-3.6
    creating build\temp.win-amd64-3.6\Release
    creating build\temp.win-amd64-3.6\Release\murmurhash
    cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Users\Nikolay\AppData\Local\Alteryx\bin\Miniconda3\include -IC:\Users\NIKOLA~1\AppData\Local\Temp\pip-build-znq14ndh\murmurhash\murmurhash\include -Ic:\users\nikolay\appdata\local\alteryx\bin\htmlplugins\pythonsdktest\include -IC:\Users\Nikolay\AppData\Local\Alteryx\bin\Miniconda3\include -IC:\Users\Nikolay\AppData\Local\Alteryx\bin\Miniconda3\include /EHsc /Tpmurmurhash/mrmr.cpp /Fobuild\temp.win-amd64-3.6\Release\murmurhash/mrmr.obj /Ox /EHsc
    error: command 'cl.exe' failed: No such file or directory

    ----------------------------------------
Command "c:\users\nikolay\appdata\local\alteryx\bin\htmlplugins\pythonsdktest\scripts\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\NIKOLA~1\\AppData\\Local\\Temp\\pip-build-znq14ndh\\murmurhash\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\NIKOLA~1\AppData\Local\Temp\pip-1rqp36vy-record\install-record.txt --single-version-externally-managed --compile --install-headers c:\users\nikolay\appdata\local\alteryx\bin\htmlplugins\pythonsdktest\include\site\python3.6\murmurhash" failed with error code 1 in C:\Users\NIKOLA~1\AppData\Local\Temp\pip-build-znq14ndh\murmurhash\
You are using pip version 9.0.1, however version 18.0 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

I had the same problem with pycham, that's why I moved to Ubuntu.

 

Cheers

Nikolay

 

 

Nikolay
6 - Meteoroid

Hi PaulN

thank you for the fast Reply. I already installed nltk with pip, but when I run pip install spacy I'm getting in the end:

Installing collected packages: numpy, murmurhash, cymem, preshed, msgpack, msgpack-numpy, toolz, cytoolz, wrapt, plac, tqdm, pyreadline, dill, thinc, ujson, regex, idna, chardet, urllib3, certifi, requests, spacy
  Running setup.py install for murmurhash ... error
    Complete output from command c:\users\nikolay\appdata\local\alteryx\bin\htmlplugins\pythonsdktest\scripts\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\NIKOLA~1\\AppData\\Local\\Temp\\pip-build-znq14ndh\\murmurhash\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\NIKOLA~1\AppData\Local\Temp\pip-1rqp36vy-record\install-record.txt --single-version-externally-managed --compile --install-headers c:\users\nikolay\appdata\local\alteryx\bin\htmlplugins\pythonsdktest\include\site\python3.6\murmurhash:
    running install
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-3.6
    creating build\lib.win-amd64-3.6\murmurhash
    copying murmurhash\about.py -> build\lib.win-amd64-3.6\murmurhash
    copying murmurhash\__init__.py -> build\lib.win-amd64-3.6\murmurhash
    creating build\lib.win-amd64-3.6\murmurhash\tests
    copying murmurhash\tests\test_import.py -> build\lib.win-amd64-3.6\murmurhash\tests
    copying murmurhash\tests\__init__.py -> build\lib.win-amd64-3.6\murmurhash\tests
    copying murmurhash\mrmr.pyx -> build\lib.win-amd64-3.6\murmurhash
    copying murmurhash\mrmr.pxd -> build\lib.win-amd64-3.6\murmurhash
    copying murmurhash\__init__.pxd -> build\lib.win-amd64-3.6\murmurhash
    creating build\lib.win-amd64-3.6\murmurhash\include
    creating build\lib.win-amd64-3.6\murmurhash\include\murmurhash
    copying murmurhash\include\murmurhash\MurmurHash2.h -> build\lib.win-amd64-3.6\murmurhash\include\murmurhash
    copying murmurhash\include\murmurhash\MurmurHash3.h -> build\lib.win-amd64-3.6\murmurhash\include\murmurhash
    running build_ext
    building 'murmurhash.mrmr' extension
    creating build\temp.win-amd64-3.6
    creating build\temp.win-amd64-3.6\Release
    creating build\temp.win-amd64-3.6\Release\murmurhash
    cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Users\Nikolay\AppData\Local\Alteryx\bin\Miniconda3\include -IC:\Users\NIKOLA~1\AppData\Local\Temp\pip-build-znq14ndh\murmurhash\murmurhash\include -Ic:\users\nikolay\appdata\local\alteryx\bin\htmlplugins\pythonsdktest\include -IC:\Users\Nikolay\AppData\Local\Alteryx\bin\Miniconda3\include -IC:\Users\Nikolay\AppData\Local\Alteryx\bin\Miniconda3\include /EHsc /Tpmurmurhash/mrmr.cpp /Fobuild\temp.win-amd64-3.6\Release\murmurhash/mrmr.obj /Ox /EHsc
    error: command 'cl.exe' failed: No such file or directory

    ----------------------------------------
Command "c:\users\nikolay\appdata\local\alteryx\bin\htmlplugins\pythonsdktest\scripts\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\NIKOLA~1\\AppData\\Local\\Temp\\pip-build-znq14ndh\\murmurhash\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\NIKOLA~1\AppData\Local\Temp\pip-1rqp36vy-record\install-record.txt --single-version-externally-managed --compile --install-headers c:\users\nikolay\appdata\local\alteryx\bin\htmlplugins\pythonsdktest\include\site\python3.6\murmurhash" failed with error code 1 in C:\Users\NIKOLA~1\AppData\Local\Temp\pip-build-znq14ndh\murmurhash\
You are using pip version 9.0.1, however version 18.0 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

As I wrote before I don't think that this is a Alteryx problem, but Windows/Python problem with compilation.

 

PaulN
Alteryx Alumni (Retired)

Hi @Nikolay,

 

Thank for the feedback.

 

In this case, it seems that Microsoft Visual C++ Build Tools (https://visualstudio.microsoft.com/downloads/#build-tools-for-visual-studio-2017) are going to be required in your case.

 

I have tested with wheel file (https://www.lfd.uci.edu/~gohlke/pythonlibs/spacy‑2.0.12‑cp36‑cp36m‑win_amd64.whl) but it seems to require to Clang.exe from LLVM (https://releases.llvm.org/3.8.1/LLVM-3.8.1-win64.exe).

 

Thanks,

 

Paul Noirel

Sr Customer Support Engineer, Alteryx

 

Nikolay
6 - Meteoroid

Hi PaulN,

 

Thank you for the proposition.

 

I already tried all Microsoft Visual Build Tools 2017 - 2015 and some others, but it still cannot compile.

 

I red in many python forums that this is common Windows- Python issue.

chrisha
11 - Bolide

Make sure, you have the correct version of Build Tools installed. This version seems to be working: http://go.microsoft.com/fwlink/?LinkId=691126&fixForIE=.exe

Note, that you might need to uninstall the previous versions of the build tools. There is a known bug of Python not finding the correct libraries and thus failing to compile anything (see https://stackoverflow.com/questions/14372706/visual-studio-cant-build-due-to-rc-exe#46679777).

 

The wheel files @PaulN mentions fail for me (https://www.lfd.uci.edu/~gohlke/pythonlibs/#spacy), because thinc 6.10.2 is included and the latest spaCy version requires thinc>=6.10.3

 

Nikolay
6 - Meteoroid

Hi Christa,

 

Thank you for the reply.

I uninstalled everything, restarted and than installed these tools and again the same error.  :(

 

 

TashaA
Alteryx Alumni (Retired)

Hello @Nikolay!

 

Thanks so much for sharing your experience, we are going to look into this and report back.

 

-Tasha

TashaA
Alteryx Alumni (Retired)

@Nikolay

 

quick update, we have this logged in our system and will post back with any new developments or troubleshooting tips.

 

Thanks!