community Alteryx IO Alteryx.com
alteryx Community

Alteryx IO Resources

Explore these resources to customize and extend the power of Alteryx with SDKs, APIs, and more.

Instructions for Updating Custom Tools to Python 3.10

baileykaplan
Moderator
Moderator
Created

‎02-20-2024 09:10 AM - edited ‎03-25-2024 11:31 AM

Overview

Alteryx Designer and the Python Engine (E1) and V2 SDKs include a version of Python. For the past 2 years, this version has been v3.8.16, but Python no longer provides support for this version. Security is paramount at Alteryx and to maintain our security standards we must upgrade Designer to Python v3.10.13 (the newest supported version of Python) for the 2024.1 release.

This means that you need to upgrade Alteryx Designer tools built with the SDKs to Python v3.10.13. in order to be compatible with Designer 24.1. Refer to the below compatibility matrix to determine which versions of Python are compatible with specific versions of Designer.

 

 

Designer 23.2 or Earlier

Designer 24.1 or Later*

Python 3.8.16

Yes

No

Python 3.10.13

No

Yes
 

*A Beta version of Designer 24.1 is available via Public Preview.

The Alteryx Designer 24.1 release is planned for early May. For your custom-built SDK tools to work in Alteryx Designer 24.1 without disruption, you must update them to Python v3.10.13 and test them before you upgrade to the newest Designer version.

If you previously made your tool publicly available (for example, posted it on Community Gallery or Marketplace) you should redistribute your tool through the same channel after you update it.

Instructions for Upgrading Python

Technical Requirements

Make sure these are installed on your system:

  • Microsoft Windows 7 or Later (64-bit)

  • Alteryx Designer Version 2024.1

  • Python Version 3.10.13

  • pip (automatically installed with Python 3.10.13)

  • node 14 (node 12 to node 16 will work, but node 14 is recommended)

You can install the last 3 with conda, via this command:  

 

conda create -n <environment name> python=3.10.13 "nodejs>=12,<=16"

 

Determine if you were using the Python Engine SDK or the V2 SDK. If you are unsure, you can check the requirements-thirdparty.txt file. If this file contains a line that says ayx_python_sdk==2.3.0, then the tool was built using the V2 SDK. 

Instructions for Python Engine SDK Users

Please note that we first recommend that you investigate a conversion to our currently supported SDK version (AYX Python SDK V2). Our latest SDK automates nearly all of the following checklist items, greatly reduces boilerplate in prior SDK versions (this version included), and can support pure Python implementations of your Python code as a 3.10 Python module. Further, the AYX Python SDK V2 creates testing scaffolding and other features that greatly improve the development experience!

We ask that you refer to the documentation for Python Engine SDK while you review this checklist and subsections. In addition, per the documentation above, the SDK leaves packaging and virtual environment management to the user. As a result, it’s difficult to recommend a comprehensive upgrade path, and we can't guarantee that deprecated or otherwise EOL 3rd-party dependencies will continue to be supported in 3.10.

Upgrade Checklist

  • Update any paths configured as shown in Python Engine SDK.

  • If you were previously using a virtual environment, you should create a new one using the Python 3.10.13 version linked via the documentation steps. Then, test your code to begin validating tool dependencies.

  • Ensure any tools’ 3rd-party libraries are compatible and supported for 3.10.13. Refer to the subsection below for tips on this process.

  • If you have packaging scripts that automate the manual packaging process of the Python Engine SDK or must manually package any Python Engine SDK deliverables, review any Python dependencies, configuration, or installations that might depend on Python 3.8 and update those references.

  • Install and test tool YXIs on the latest version of Designer. At the time of writing, the latest version is 24.1.

Front End Upgrades/Updates

There are no known updates necessary for the front-end components in this case.

Python and Version Upgrades

Should you run into errors with Python versions and included libraries, check out Python’s own documentation sections for notable changes in each release:

Libraries often have similar guides or compatibility references to help you research the level of effort:
Installing PyArrow — Apache Arrow v15.0.1.

 

Instructions for V2 SDK Users

Upgrading to Python 3.10 is fairly straightforward for current V2 SDK users. Since the pre-release wheels are available on PyPI, install the pre-release SDK and CLI versions into the current Conda environment via this command:

pip install ayx_plugin_cli==1.2b0 ayx_python_sdk==2.4b0

Next, update the dependency in requirements-thirdparty.txt:

ayx_python_sdk==2.4b0

Once the beta program has ended and the full release of the Python packages is available on PyPI, install them via this command:

pip install ayx_plugin_cli==1.2.0 ayx_python_sdk==2.4.0

Last, update the dependency in requirements-thirdparty.txt:

ayx_python_sdk==2.4.0

Instructions for Redistributing the Tool

If you previously posted tools on Community Gallery, you should repost them on Community Gallery after you update their Python version. You should use this naming convention: 24.1 - TOOL NAME

The TOOL NAME should be the original name of the tool when you first posted the tool on Community Gallery. If possible, please comment on the original Community Gallery post for the tool, and link to the Community Gallery post of the upgraded version of the tool. This lets other users easily find the updated version of your tool.

Marketplace

If you have tools on the Marketplace, please submit new files for 24.1 and notify alteryxmarketplace@alteryx.com to ensure the new files are verified and published. You can add 24.1 and Python compatibility on your edition page.

 

Troubleshooting

There are 3 ways to test SDK tools—unit testing, the test client, and in Designer.

Unit Testing

The CLI generates a set of Python unit tests alongside the tool code. You can discover and execute these unit tests from IDEs like VSCode or Pycharm, or via the ayx_plugin_cli test command from the terminal. You can use these tests to test general Python issues—for instance, to catch bad indents or import errors—but you can also use them to ensure that the business logic represented by each of the callbacks is consistent. For more details on testing, go to Test Scaffolding.

Test Client

In addition to these unit tests, we also offer a separate test client that mimics a Designer-like environment. You can run this from the command line against a single standalone Python tool. You can download the latest test client from alteryx/ayx-develoepr-sdk  and find documentation on how to use it at Test Client .

Test in Designer

Once the requirements are fully up-to-date and the workspace updated, you can build the tool with ayx_plugin_cli create-yxi, or you can build and install the tool with a single command via ayx_plugin_cli designer-install. This command also has a “hot-load” developer mode, which lets you write your code, save it in your IDE, and instantly reflect those changes in Designer on the next tool run. To use this feature, install the tool with ayx_plugin_cli designer-install --dev --install-type user with the 3.10 conda environment active.

The designer-install command also runs a quick sanity-check run of the plugin after installation. If the plugin loads and starts up properly—indicating no compilation errors—the log shows something like this:

[Installing YXI] .  install_yxi
[Installing YXI] ListenPort: None
[Installing YXI] INFO: Starting Input tool with AMP Provider.
[Installing YXI] ListenPort: 54310
[Installing YXI] finished

Common Issues

When you upgrade tools from Python version 3.8 to 3.10, the most likely source of issues are dependency conflicts. This is an example of the error message formed by a dependency conflict when you install with pip:

ERROR: Cannot install pytest-cov==2.12.1 and pytest<4.6 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested pytest<4.6
    pytest-cov 2.12.1 depends on pytest>=4.6

If this happens, pin the correct version (in the above example, pytest==4.6 ) in the requirements-thirdparty.txt, and rebuild the tool.

If you encounter an NPM issue, this is a problem in the UI component. Try to run npm run build in the offending UI subdirectory, and see what happens.

If you encounter an issue like the one below when you run the unit tests or in the sanity-check run step of designer-install, this means there are import errors somewhere. This might be a missing dependency or a badly named package. In the case of the former, add that to one of the requirements.txt files, and in the case of the latter, follow the traceback to the misspelled name.

EXCEPTION: ERROR: Couldn't find module ayx_plugins for loading plugins.
 Traceback (most recent call last):
   File "c:\ayx-core\ayxsdk\ayxpythonsdk\ayx_python_sdk\providers\amp_provider\plugin_class_loader.py", line 62, in load_plugin_class
     plugins_module = importlib.import_module(plugins_package)
   File "C:\Users\aneesh.nazar\.conda\envs\dev_310\lib\importlib\__init__.py", line 126, in import_module
     return _bootstrap._gcd_import(name[level:], package, level)
   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
   File "<frozen importlib._bootstrap_external>", line 883, in exec_module
   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
   File "c:\ayx-core\ayxsdk\.doit.example_tools\basetools\backend\ayx_plugins\__init__.py", line 22, in <module>
     from .conversion_passthrough import ConversionPassthrough
   File "c:\ayx-core\ayxsdk\.doit.example_tools\basetools\backend\ayx_plugins\conversion_passthrough.py", line 21, in <module>
     import bar
 ModuleNotFoundError: No module named 'bar'

 During handling of the above exception, another exception occurred:

 Traceback (most recent call last):
   File "C:\Users\aneesh.nazar\AppData\Roaming\Alteryx\Tools\Input_2_0\main.pyz", line 48, in <module>
     start_sdk_tool_service(tool_package, tool_name, os.getenv("TOOL_SERVICE_ADDRESS"))
   File "c:\ayx-core\ayxsdk\ayxpythonsdk\ayx_python_sdk\providers\amp_provider\__main__.py", line 48, in start_sdk_tool_service
     plugin_class = load_plugin_class(plugins_package, tool_name)
   File "c:\ayx-core\ayxsdk\ayxpythonsdk\ayx_python_sdk\providers\amp_provider\plugin_class_loader.py", line 64, in load_plugin_class
     raise ModuleNotFoundError(
ModuleNotFoundError: ERROR: Couldn't find module ayx_plugins for loading plugins.

If you encounter other issues during installation, here are a few things to try: delete the .ayx_cli.cache folder, or create a fresh conda environment, reinstall the CLI and SDK, and rerun the commands.

Once you install the tool, open/reopen Designer, and add the tool to the canvas. If you encounter something like the below error, you’re using an older version of Designer. Python version 3.10 tools only work in the 2024.1 beta version of Designer.

Error: sdkPassthroughTool (3): Version 3.10 was not found in our alteryx distribution. Please select another version.
Contributors