Building the Igor CLI with Click
This summer, I am building an IPMI management console for the OSUOSL. IPMI is the interface implemented by special hardware that lets you command a machine over a network as long as it is plugged in, even if the machine is powered off or refuses to boot an OS. If you have ever had ops engineers rebooting a bricked datacenter server with a magic console, you have seen IPMI.
There are some requirements to working with IPMI: you need to have commands such as
available on your OS, and you need to know the IPMI credentials for the machine you want to access.
If your datacenter machine has many IPMI users, or if they would like access via smartphones and web
browsers, you quickly hit some limitations. Sharing the single IPMI username and
password for the machine with all your users is a bad idea. Additionally,
equivalents for mobile OS’s are few and brittle.
I planned to address this with the following scheme: implement a REST API
ipmitool commands, and then have very thin CLI, web GUI and Android clients that
consume this REST API. These thin clients interface with the REST API to manage users, machines and
user-machine permissions in addition to actually performing the IPMI operations.
Here I discuss the CLI design and implementation with a recent Python CLI framework called Click, by Armin Ronacher of Flask, Werkzeug and itsdangerous fame. While it may seem like a natural choice once you are done reading this post, note that I had to throw away a bunch of code after a false start with a different but way more popular CLI framework. Before I get into that, let us look at some of the goals I had for the CLI.
If you have used the
heroku CLI, you already know what this means. Since a termshow is
worth a thousand man-pages, here is one below:
The workflow above shows a new Igor user who wants to view the available machines. She knows
nothing about Igor, and begins by typing in the command itself. She then navigates to the
command and finds the
list subcommand, and is then directed towards authorizing herself with the
auth command and finally viewing the list of machines.
This demonstrates some important features that I wanted Igor to have.
Explorability. Nobody likes reading a monochrome wall of text that is the typical manpage. A sensible, simple hierarchy with up-to-date documentation and helpful error messages is enough for the average user to navigate the average CLI, just like how you would explore a new website. An added bonus would be friendly and forgiving prompts for when the user forgets to provide any required options.
Extensibility. APIs change constantly. Updating your CLI to match a new API endpoint should not require touching too many files. Ideally, you would just add a single new Python module for a new command that also contains all its subcommands, and have everything else in the hierarchy fall into place.
DRYness. Subcommands may share the need for certain data that could be handled by
their common parent instead of in each of them individually. A common case is having a
--verbose option: instead of being parsed by every subcommand, have the root command parse
and store this in a configuration object that is available to all subcommands.
An additional aesthetic requirement is that the CLI mirror the hierarchical structure of the REST API, so users of both can switch contexts easily.
How would one go about quickly implementing such a design? A probable first step would be looking at the Python Guide for recommendations and figuring out what the most popular CLI framework out there is.
A False Start
If you started this way, you would come across and immediately fall for the incredibly sexy docopt. To see why, here is an example of parsing CLI options with docopt:
docopt constructs argument parsing rules from your docstring. Hence, you end up with well-written, up-to-date documentation for your commands, and your code remains minimal. To complete things, docopt comes with many examples, including one partially implementing the complex git CLI.
However, you simply cannot implement the hierarchy I described above without
a lot of additional effort. I personally think the provided git example is
git help <command>, the first hierarchical composition of commands
I wanted to try, does not work. I spent some time on this and concluded
that it is non-trivial to try and get it to work either.
I hated throwing away this tiny method dressed in a beautiful docstring that takes care of everything for you. But sometimes things just don’t work out.
Click satisfied the aforementioned goals perfectly. It is a solid, well-designed library that is similar to Flask in its excellent documentation, and abundance of Python decorators. In short, it had what I needed:
- Arbitrarily nested commands with minimal code.
- Automatic help-page generation.
- Prompts for required but unprovided options.
- Sharing data between commands via a shared object.
Let us look at some of the more interesting internals of the Igor CLI implementation.
The Igor REST API uses token-based authentication: you first request an API token by providing your username and password, and all subsequent requests must contain this API token. The API token has an expiry date and is generally a better idea than sending your username and password with every API request.
For credential management, the Igor CLI again follows the lead of
Once a user successfully logs in using
igor auth login,
the username and API token are stored in
~/.netrc, which is a
standard flat-file used for credential storage. It looks like this:
The netrc Python module helps you read the
~/.netrc file into a Python dictionary, but
unlike the eponymous Ruby gem, it does not provide a way to save dictionaries back to
~/.netrc files. I had to write some helper methods to do this.
Why store credentials in this file? Being a pretty standard storage
location, many other tools use this as the default source of credentials, like FTP and,
importantly for us, the Python requests module that eases working with HTTP in Python.
It automatically picks up credentials from
~/.netrc and provides
them to the correct host. Hence, by having these credentials stored in
we could avoid additional code anywhere else that retrieves and sends them across with
A natural question now would be: what about validation? What if the requests module does not
find any stored credentials? What happens if the user does something strange; for
example, setting a machine’s power state to
monkey instead of the valid
The Igor CLI is a thin layer over the Igor REST API, which is a thin layer over
appears really fragile; if each layer performs its own argument parsing and validation, doing
something like adding new argument parameters would require changing each layer.
The strategy I adopted is to aggressively delegate error handling, and enforce some invariants.
The Igor CLI uses a single
make_api_request method defined here. If the user is not logged
in or is not authorized to access a machine, the function fails with a pointer to
igor auth or
igor permissions. For any other API response apart from HTTP 200 OK, the method bails out
immediately, printing the API response. So the CLI subcommands themselves are simply messengers,
performing no validation themselves.
As an example, consider the
ipmitool power command that takes exactly one of
state parameter. The Igor CLI takes the
--state argument and sends it to the Igor
REST API. The REST API in turn passes it on to
ipmitool via the pyipmi module, which raises
IpmiError if there is anything wrong. If such an exception occurs, the API returns
a message (with a HTTP 400 error), which is then displayed by the CLI’s
With this strategy, if
ipmitool begins accepting a new
monkey power state, we would not need any
code changes at all!
The Igor CLI is easily installable using pip:
It currently allows one to add, remove, view and update users, machines and user-machine permission pairs. It also allows one to view and set a machine’s power state. The remaining IPMI operations are work in progress. A particularly interesting one is interacting with the serial-on-LAN console; would it make sense to create some sort of persistent duplex HTTP connection? Also coming up are the ncurses, web and Android clients. Further details are available in my project proposal and timeline.
This appendix provides termshows demonstrating some of the less interesting command groups that
were not discussed in the post above. All commands were executed as Igor user
The Igor REST API server was running locally. Instead of passing the
option with every command, the following is placed in
See pull request.
Machine Management Operations
See pull request.
User Management Operations
See pull request.
Permission Management & Power State Operations
Permission management: See pull request.
IPMI power state: See pull request.