Using Python 3.8.0 Azure Runbooks with Python Packages

Azure Python Runbooks support both Python 2.7.12 as well as Python 3.8.0. In this blog, we’ll look at leveraging Python 3.8.0 both when running Python Runbooks leveraging Azure Compute as well as leveraging a Hybrid Runbook Worker when using Python installed on Windows Server.

Importing Python Packages and Dependencies

The runbook we want to leverage uses third-party python packages as can be seen from the following code:

#!/usr/bin/env python3
print("Hello World!")

from nslookup import Nslookup

domain = "www.shudnow.io"

# Initialize Nslookup
dns_query = Nslookup()
# Alternatively, the Nslookup constructor supports optional
# arguments for setting custom dns servers (defaults to system DNS),
# verbosity (default: True) and using TCP instead of UDP (default: False)
dns_query = Nslookup(dns_servers=["1.1.1.1"], verbose=False, tcp=True)

ips_record = dns_query.dns_lookup(domain)
print(ips_record.response_full, ips_record.answer)

The Python Module we will be leveraging is Nslookup which can be downloaded via nslookup · PyPI. In our Runbook, if we go to Python Packages, we can see there is an option to add a Python Package.

This provides us the ability to add a .whl or .tar.gz downloaded Python Package file.

Although this option exists, what happens if our module has depdencies on other Python Packages? There is a Runbook that can be used to install a Python Package into your Automation Account including all dependent packages. This is documented here: Import a package with dependencies | Microsoft Docs which links to the import_py3package_from_pypi.py Runbook that exists in GitHub.

In this Runbook, we see there are several parameters:

Args:
    subscription_id (-s) - Subscription id of the Automation account
    resource_group (-g) - Resource group name of the Automation account
    automation_account (-a) - Automation account name
    module_name (-m) - Name of module to import from pypi.org
    Imports module
    Example:
        import_python3package_from_pypi.py -s xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx -g contosogroup -a contosoaccount -m pytz

I created a Runbook and copied in the Raw GitHub Code from the import_python3package_from_pypi.py script. In Start Runbook, I specified the Parameters as follows based on the Args specified within the import_python3package_from_pypi.py script.

Once you click Ok to start the Runbook, in Packages, you’ll see 2 packages have been added. Using the Runbook to Import Python Packages guarantees that your package will be imported with any necessary dependencies.

After a short period of time, your packages should show as Available.

Executing the Runbook leveraging Azure Compute

We created our other Runbook leveraging our code to do a DNS Lookup and saved our Runbook as seen here:

Click the Test Pane and execute the Runbook using Azure Compute. We see that our Runbook was successfully executed using our third-party Python Modules.

Executing the Runbook leveraging Hybrid Runbook Worker

The scope of this article isn’t to go over how to set up a Hybrid Runbook Worker. For that, please see the following article: Azure Automation Hybrid Runbook Worker overview | Microsoft Docs. The basic idea of leveraging a Hybrid Runbook Worker is that you leverage your own Virtual Machine to execute the Runbook. That can be an Azure VM or a VM outside of Azure which does require the VM to be Arc-Enabled.

We have one Hybrid Runbook Worker Group containing one Azure VM.

I downloaded Python 3.8.0 from the following URL: Python Release Python 3.8.0 | Python.org. What we’ll notice is that if you install Python 3.8.0 with the default install location, it will not work even if you choose the option to set the PATH as part of the install. Let’s take a look.

I’m installing Python 3.8.0 as an administrator and select the option to “Add Python 3.8 to PATH.”

After installation, I see Python is successfully added to the PATH Environmental Variable. When choosing the default install location, the Python 3.8.0 installer leverages Path within the User variables.

What we see is that it is looking for Python in a specific directory which is not the same directory the Python installation software uses. Therefore, you’ll want to uninstall Python 3.8.0 and re-install using “C:\WPy64-3800\python-3.8.0.amd64” as the installation directory. When installing, choose Customize Installation and choose the following Advanced Options.

After installation, we see the new environmental variables have been added to the custom location of our install. When choosing a custom install location and also choosing “Install for all users”, the Python 3.8.0 installer leverages Path within the System variables.

If we go to re-run the runbook, we can see we no longer have the same error. However, we have a new error: No module named ‘nslookup’ which indicates that the Hybrid Runbook Worker does not leverage/download the Python Packages installed in our Automation Account. Instead, we need to install the modules on our Hybrid Runbook Worker.

Let’s see what happens if we try creating a Python Virtual Environment. As we are on Windows, we’ll run the following commands to create the Virtual Environment, activate it, and install our two Python Packages.

py -m venv runbook
.\runbook\Scripts\activate
py -m pip install nslookup

The following screenshot provides the results of running all 3 commands.

However, after running the Runbook against the Hybrid Worker, we get the same results.

As can be seen, executing the Runbook within a Virtual Environment that has our Python Packages installed does not work. Therefore, let’s deactivate our Virtual Environment and re-install our Packages to the OS-based Python Install.

Now that the nslookup and dependent dnspython Python Packages have been installed, let’s again re-run our Runbook against our Hybrid Worker.

Voila! We have a successful execution.