Why Use `pip install --user`?

When working with Python, you're likely familiar with the process of installing packages using the popular package manager, pip. It's a powerful tool that simplifies the process of adding third-party libraries to your Python environment. However, you might have come across the --user flag while installing something with pip and wondered what its purpose is and why it might matter. In this blog post, we'll unravel the mysteries of pip install --user and explore its significance in plain English.

Unraveling the `--user` flag

Before we dive in, let's clarify what the pip install --user command does. According to pip install --help, it instructs pip to:

--user  Install to the Python user install directory for your platform.
        Typically ~/.local/, or %APPDATA%\Python on Windows. (See the
        Python documentation for site.USER_BASE for full details.)

In simpler terms, using pip install --user changes the default location for Python package installations. Instead of installing packages in system-wide directories, which often require administrative privileges, this flag installs packages in a user-specific directory, safely tucked away from system-wide locations.

But why does this matter?

Installation Locations: System-Wide vs. User-Specific

The default behavior of pip without the --user flag places packages in system-wide directories like /usr/local/lib/python${PY_MAJOR}/dist-packages on Unix-like systems. While this approach makes packages accessible to all users on a machine, it comes with a significant drawback — the need for administrative access.

Here's where the --user flag makes the difference. When you use pip install --user cowsay, it directs pip to install packages in a user-specific location, typically ~/.local/lib/python${PY_MAJOR}.${PY_MINOR}/site-packages. This not only allows you to install packages without administrative rights but also ensures that your installed packages remain isolated to your user account, safeguarding against conflicts and giving you full control over your Python environment. These user-specific packages remain invisible to other users who access the same host.

The power of the --user flag extends beyond pip install. It also influences other pip commands like list, aligning their behavior with the user-specific installation location. For example, pip list --user will display only packages installed with pip install --user.

In addition to using the --user flag directly in your pip commands, there's another convenient trick for those who prefer user-specific installations frequently: setting the PIP_USER environment variable. By configuring PIP_USER=1 in your environment, you instruct pip to default to the --user option for all installations, eliminating the need to specify the flag each time.

No Virtual Environments? No Problem!

Using pip install --user narrows the scope of the current pip command to operate within the local Python package installation of your user account, rather than the default system-wide location. This distinction is crucial in multi-user setups. Anything installed in the system location becomes visible to all users, potentially leading to version conflicts if the package you install has dependencies needed by others. So, opting for the --user flag is a wise choice, especially for packages intended to be available to all users by default, without the need for root/sudo access or admin privileges.

If you're the sole user of your machine, the difference between user and system locations is minimal. The former simply places packages in a different spot, which may or may not need to join your system's PATH, depending on the package and its purpose. Some packages come with command-line tools that must be added to the PATH for accessibility from your terminal.

To find your installation spot, you can use some Python magic:

import site
print(site.USER_SITE)

You can even customize it by setting the PYTHONUSERBASE environment variable, updating the value of site.USER_BASE. For example, if you want to install some_pkg into an environment with a customized site.USER_BASE of /myappenv, how you do it:

export PYTHONUSERBASE=/myappenv
python -m pip install --user cowsay

The Role of Virtual Environments

However, the --user flag is not valid inside a virtual environment's pip (the default kind created with the --no-site-packages flag). It simply doesn't make sense in that context. If you attempt to use --user within an active venv/virtualenv, you'll encounter an error like this:

ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.

If you create a virtual environment with the --system-site-packages argument, the package installed with the --user flag will be placed in the user-specific directory as expected. However, because the virtual environment inherits access to the system site packages, it may also see the package installed with --user in the system site packages, potentially leading to conflicts or confusion if the package versions differ between the system and user-specific locations.

Conclusion

In summary, the pip install --user command is a powerful tool for managing Python packages in your personal environment. It allows you to install packages without administrative hassles, keeps your package installations safely isolated, and plays a pivotal role in multi-user settings and virtual environments.

However, if you seek ultimate flexibility and want to avoid potential version conflicts with each package installation, consider using virtualenv. It bypasses the complexity of --user and ensures a seamless Python experience.