Simplifying terminal operations with python modules

Table of contents

Almost any linux distribution will ship with python3 installed by default nowadays. With the numerous improvements that the python language saw over time, one has been flying under the radar of most users: modules that can be executed directly from the command line.

Opening a URL in the default web browser

If you need to open a URL in the default web browser, the webbrowser module can do that:

python3 -m webbrowser http://example.com/test

If no browser window is currently open, one will be started. If one is already open, the URL will be visited in a new tab. If you want to always load the URL in a new window, you can use the -n flag:

python3 -m webbrowser -n http://example.com/test

While not necessarily useful on it's own, it can be a great way to streamline behavior when used from shell scripts without relying on other tools than the python interpreter being available on the system.

Validating and formatting JSON

When working with JSON data, it is often desirable to format minified documents from dumps or api responses to improve readability. The python interpreter can handle this task through the json.tool module:

echo '{"name": "jonathan", "age": 51, "hobbies": ["swimming", "biking", "hiking"]}' | python3 -m json.tool

When no other argument are provided, json.tool will read JSON data from stdin and print a formatted version to stdout:

{
   "name": "jonathan",
   "age": 51,
   "hobbies": [
       "swimming",
       "biking",
       "hiking"
   ]
}

File support is also built into the module, for example reading from minified.json and writing the formatted JSON document to readable.json:

python3 -m json.tool minified.json readable.json

Formatting preferences can be adjusted through flags, for example by using --indent 2 to switch indentation to use two spaces, or --tab to use tabs instead of spaces for indentation.

Running an HTTP server

The builtin http.server module can be used to serve a directory over HTTP:

python3 -m http.server --directory /data

By default, the contents of the /data directory will not be served at 0.0.0.0:8000, until the process is stopped with ctrl+c. The incoming requests will be logged to the terminal while the server runs:

Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
127.0.0.1 - - [26/Apr/2024 04:44:51] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [26/Apr/2024 04:45:01] "GET /data.json HTTP/1.1" 200 -

While this is already quite handy, there is one more feature: CGI support. Running the http server with the --cgi flag will enable the server to automatically execute scripts and programs in the cgi-bin/ directory in the webroot. Let's assume the following directory structure at /webroot:

├── cgi-bin
│  └── hello.py
└── data.json

The contents of hello.py can be anything, as long as it produces an HTTP response:

#!/usr/bin/env python3

print("""200 OK HTTP/1.1
Content-Type: text/html

<!Doctype html>
<title>Hello</title><h2>hello visitor</h2>
""")

Make sure the file is executable:

chmod +x cgi-bin/hello.py

And finally, run the server with the --cgi flag:

python3 -m http.server --directory /webroot --cgi

The server is available at http://0.0.0.0:8000 again. But while it will serve most files normally, accessing a script inside cgi-bin/ will execute it instead. Accessing http://0.0.0.0:8000/cgi-bin/hello.py will show the html greeting page instead of the python source code. The CGI scripts can be written in any language, as long as they are executable and produce a valid HTTP response.

Sharing a local directory over FTP

While this module is not part of the builtin python3 modules, it is valuable enough to mention in this list: pyftpdlib. This module can quickly start an FTP server to allow remote access to a local directory, with optional authentication. It can be installed through pip or your distribution's package manager, if global python modules are handled by that:

pip install pyftpdlib # for systems where pip handles global modules
apt install python3-pyftpdlib # for debian-based distros

Once installed, it can be invoked from the command line:

python3 -m pyftpdlib -d /data

This starts an FTP server on port 2121, where users can access the contents of the local /data directory in read-only mode without login. The server will run until the terminal window is closed or the process stopped with ctrl+c.

FTP servers are started read-only by default, and only adding the -w flag will allow connected users to write to the exposed directory.

A more complex use case requires only a few adjustments:

python3 -m pyftpdlib -d /data -p 21 -u myUser -P pass123 -w

The resulting FTP server still exposes the contents of /data, but now runs on the default FTP port 21, allows writing by connected clients and requires authentication as user myUser with password pass123.

More articles

A complete guide to running windows software on linux

Making windows programs behave like they were written for linux

Essential SSH commands

Securely managing remote servers

How software compression works

Making files smaller without losing information

Setting up an nginx load balancer

Distributing requests to multiple servers