Sabyasachi Samaddar walks through what I consider a real difficulty:
It is really challenging when you need to install third-party .whl packages into a DEP-enabled Azure Synapse Spark Instance.
According to the documentation, https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-azure-portal-add-librar… Installing packages from PyPI is not supported within DEP-enabled workspaces. Hence we cannot just upload the .whl packages into the workspace. We need to upload all the dependencies along with the .whl package and it will be an offline installation. Now Synapse spark clusters come with in-built packages and hence we may find some conflicts when we try to install some third-party packages.
Read on to see what you need to do.