Python Dependency Manager Companion

# Working with PEP 582 !!! warning "PEP 582 has been rejected" This is a rejected PEP. However, due to the fact that this feature is the reason for PDM's birth, PDM will retain the support. We recommend using [virtual environments](./venv.md) instead. With [PEP 582](https://www.python.org/dev/peps/pep-0582/), dependencies will be installed into `__pypackages__` directory under the project root. With [PEP 582 enabled globally](#enable-pep-582-globally), you can also use the project interpreter to run scripts directly. **When the project interpreter is a normal Python, this mode is enabled.** Besides, on a project you work with for the first time on your machine, if it contains an empty `__pypackages__` directory, PEP 582 is enabled automatically, and virtualenv won't be created. ## Enable PEP 582 in projects managed my pdm To make pdm use PEP 582 instead of virtual environment, set `python.use_venv` config variable to False: ```bash pdm config python.use_venv False ``` ## Enable PEP 582 globally To make the Python interpreters aware of PEP 582 packages, one needs to add the `pdm/pep582/sitecustomize.py` to the Python library search path. === "Windows" One just needs to execute `pdm --pep582`, then environment variable will be changed automatically. Don't forget to restart the terminal session to take effect. === "Mac and Linux" The command to change the environment variables can be printed by `pdm --pep582 [<SHELL>]`. If `<SHELL>` isn't given, PDM will pick one based on some guesses. You can run `eval "$(pdm --pep582)"` to execute the command. You may want to write a line in your `.bash_profile`(or similar profiles) to make it effective when logging in. For example, in bash you can do this: ```bash pdm --pep582 >> ~/.bash_profile ``` Once again, Don't forget to restart the terminal session to take effect. ??? note "How is it done?" Thanks to the [site packages loading](https://docs.python.org/3/library/site.html) on Python startup. It is possible to patch the `sys.path` by executing the `sitecustomize.py` shipped with PDM. The interpreter can search the directories for the nearest `__pypackage__` folder and append it to the `sys.path` variable. ## Configure IDE to support PEP 582 Now there are no built-in support or plugins for PEP 582 in most IDEs, you have to configure your tools manually. ### PyCharm Mark `__pypackages__/<major.minor>/lib` as [Sources Root](https://www.jetbrains.com/help/pycharm/configuring-project-structure.html#mark-dir-project-view). Then, select as [Python interpreter](https://www.jetbrains.com/help/pycharm/configuring-python-interpreter.html#interpreter) a Python installation with the same `<major.minor>` version. Additionally, if you want to use tools from the environment (e.g. `pytest`), you have to add the `__pypackages__/<major.minor>/bin` directory to the `PATH` variable in the corresponding run/debug configuration. ### VSCode Add the following two entries to the top-level dict in `.vscode/settings.json`: ```json { "python.autoComplete.extraPaths": ["__pypackages__/<major.minor>/lib"], "python.analysis.extraPaths": ["__pypackages__/<major.minor>/lib"] } ``` This file can be auto-generated with plugin [`pdm-vscode`](https://github.com/frostming/pdm-vscode). [Enable PEP582 globally](#enable-pep-582-globally), and make sure VSCode runs using the same user and shell you enabled PEP582 for. ??? note "Cannot enable PEP582 globally?" If for some reason you cannot enable PEP582 globally, you can still configure each "launch" in each project: set the `PYTHONPATH` environment variable in your launch configuration, in `.vscode/launch.json`. For example, to debug your `pytest` run: ```json { "version": "0.2.0", "configurations": [ { "name": "pytest", "type": "python", "request": "launch", "module": "pytest", "args": ["tests"], "justMyCode": false, "env": {"PYTHONPATH": "__pypackages__/<major.minor>/lib"} } ] } ``` If your package resides in a `src` directory, add it to `PYTHONPATH` as well: ```json "env": {"PYTHONPATH": "src:__pypackages__/<major.minor>/lib"} ``` ??? note "Using Pylance/Pyright?" If you have configured `"python.analysis.diagnosticMode": "workspace"`, and you see a ton of errors/warnings as a result. you may need to create `pyrightconfig.json` in the workspace directory, and fill in the following fields: ```json { "exclude": ["__pypackages__"] } ``` Then restart the language server or VS Code and you're good to go. In the future ([microsoft/pylance-release#1150](https://github.com/microsoft/pylance-release/issues/1150)), maybe the problem will be solved. ??? note "Using Jupyter Notebook?" If you wish to use pdm to install jupyter notebook and use it in vscode in conjunction with the python extension: 1. Use `pdm add notebook` or so to install notebook 2. Add a `.env` file inside of your project directory with contents like the following: ``` PYTHONPATH=/your-workspace-path/__pypackages__/<major>.<minor>/lib ``` If the above still doesn't work, it's most likely because the environment variable is not properly loaded when the Notebook starts. There are two workarounds. 1. Run `code .` in Terminal. It will open a new VSCode window in the current directory with the path set correctly. Use the Jupyter Notebook in the new window 2. If you prefer not to open a new window, run the following at the beginning of your Jupyter Notebook to explicitly set the path: ``` import sys sys.path.append('/your-workspace-path/__pypackages__/<major>.<minor>/lib') ``` > [Reference Issue](https://github.com/pdm-project/pdm/issues/848) ??? note "PDM Task Provider" In addition, there is a [VSCode Task Provider extension][pdm task provider] available for download. This makes it possible for VSCode to automatically detect [pdm scripts][pdm scripts] so they can be run natively as [VSCode Tasks][vscode tasks]. [vscode tasks]: https://code.visualstudio.com/docs/editor/tasks [pdm task provider]: https://marketplace.visualstudio.com/items?itemName=knowsuchagency.pdm-task-provider [pdm scripts]: scripts.md ### Neovim If using [neovim-lsp](https://github.com/neovim/nvim-lspconfig) with [pyright](https://github.com/Microsoft/pyright) and want your `__pypackages__` directory to be added to the path, you can add this to your project's `pyproject.toml`. ```toml [tool.pyright] extraPaths = ["__pypackages__/<major.minor>/lib/"] ``` ### Emacs You have a few options, but basically you'll want to tell an LSP client to add `__pypackages__` to the paths it looks at. Here are a few options that are available: #### Using `pyproject.toml` and pyright Add this to your project's `pyproject.toml`: ```toml [tool.pyright] extraPaths = ["__pypackages__/<major.minor>/lib/"] ``` #### eglot + pyright Using [pyright](https://github.com/microsoft/pyright) and [eglot](https://github.com/joaotavora/eglot) (included in Emacs 29), add the following to your config: ```emacs-lisp (defun get-pdm-packages-path () "For the current PDM project, find the path to the packages." (let ((packages-path (string-trim (shell-command-to-string "pdm info --packages")))) (concat packages-path "/lib"))) (defun my/eglot-workspace-config (server) "For the current PDM project, dynamically generate a python lsp config." `(:python\.analysis (:extraPaths ,(vector (get-pdm-packages-path))))) (setq-default eglot-workspace-configuration #'my/eglot-workspace-config) ``` You'll want pyright installed either globally, or in your project (probably as a dev dependency). You can add this with, for example: ```bash pdm add --dev --group devel pyright ``` #### LSP-Mode + lsp-python-ms Below is a sample code snippet showing how to make PDM work with [lsp-python-ms](https://github.com/emacs-lsp/lsp-python-ms) in Emacs. Contributed by [@linw1995](https://github.com/pdm-project/pdm/discussions/372#discussion-3303501). ```emacs-lisp ;; TODO: Cache result (defun linw1995/pdm-get-python-executable (&optional dir) (let ((pdm-get-python-cmd "pdm info --python")) (string-trim (shell-command-to-string (if dir (concat "cd " dir " && " pdm-get-python-cmd) pdm-get-python-cmd))))) (defun linw1995/pdm-get-packages-path (&optional dir) (let ((pdm-get-packages-cmd "pdm info --packages")) (concat (string-trim (shell-command-to-string (if dir (concat "cd " dir " && " pdm-get-packages-cmd) pdm-get-packages-cmd))) "/lib"))) (use-package lsp-python-ms :ensure t :init (setq lsp-python-ms-auto-install-server t) :hook (python-mode . (lambda () (setq lsp-python-ms-python-executable (linw1995/pdm-get-python-executable)) (setq lsp-python-ms-extra-paths (vector (linw1995/pdm-get-packages-path))) (require 'lsp-python-ms) (lsp)))) ; or lsp-deferred ```