Plugins: import your processors

We are almost half-way through our tutorial on MAFw. We have learned what a Processor is and how we can create our own processors just by subclassing the base class. With the ProcessorList we have seen an easy way to chain the execution of many processors. So by now, you might be tempted to open your IDE and start coding stuff… But please hold on your horses for another minute, MAFw has much more to offer.

In the previous pages of this tutorial, we always coded our processors and then executed them manually, either from the python console or via an ad-hoc script. This is already helpful, but not really practical, because everytime you want to change which processors are executed or one of their parameters, we need to change the script code. This seems very likely the job for a configuration file, doesn’t it?

MAFw is providing you with exactly this capability, a generic script that will read your configuration file (steering file in the MAFw language) and execute it. Here comes the problem, how can the execution framework load your |processors| since they are not living in the same library?

To this problem there are two solutions: a quick and dirty and a clean and effective! Of course you will have the tendency to prefer the first one, but really consider the benefits of the second one before giving up reading!

The quick and dirty: code inside MAFw

When you install MAFw on your system, you can always do it in development mode (for pip this corresponds to the -e option). In this way, you will be allowed to code your processors library directly inside your locally installed MAFw.

It is very quick, because you can start coding right away, but it is also rather dirty, because it will be much harder to install MAFw updates and after a while the project will become too big and messy.

For these and many other reasons, we are convinced that this is not the right way to go.

The clean and effective: turn your library into a plugin

It may seem more challenging than it actually is, but in reality, it is quite simple. MAFw uses the plugin system developed by pluggy, that is very powerful and at the same time relatively easy to deploy.

After you have installed MAFw in your system (you can do it in development mode if you want to contribute!), create a new project to store your processor. How to create a project is well described in details here with a step by step guide. You don’t need to upload your package tp PyPI, you can stop as soon as you are able to build a wheel of your package and install it in a virtual environment.

To achieve this, you would need to have a pyproject.toml file, with the project metadata, the list of dependencies and other things. Here below is an example of what you should have:

Listing 10 An example of pyproject.toml for a plugin
 1[build-system]
 2requires = ["hatchling"]
 3build-backend = "hatchling.build"
 4
 5[project]
 6name = "fantastic_analysis"
 7dynamic = ["version"]
 8description = 'My processor library'
 9readme = "README.md"
10requires-python = ">=3.8"
11license = "MIT"
12keywords = []
13authors = [
14  { name = "Surname Name", email = "this.is.me@my.domain.com" },
15]
16classifiers = [
17  "Development Status :: 4 - Beta",
18  "Programming Language :: Python",
19  "Programming Language :: Python :: 3.8",
20  "Programming Language :: Python :: 3.9",
21  "Programming Language :: Python :: 3.10",
22  "Programming Language :: Python :: 3.11",
23  "Programming Language :: Python :: 3.12",
24  "Programming Language :: Python :: Implementation :: CPython",
25  "Programming Language :: Python :: Implementation :: PyPy",
26]
27
28dependencies = ['mafw'] # plus all your other dependencies
29
30[project.urls]
31Documentation = "https://github.com/..."
32Issues = "https://github.com/..."
33Source = "https://github.com/."
34
35## THIS IS THE KEY PART
36## --------------------
37# you can add as many lines you want to the table.
38# always use unique names for the entry.
39
40[project.entry-points.'mafw']
41fantastic_analysis_plugin = 'fantastic_analysis.plugins'
42
43[tool.hatch.version]
44path = "src/fantastic_analysis/__about__.py"

Particularly important are lines 40 and 41. There you declare that when installed in an environment, your package is providing a plugin for MAFw, in particular this plugin that you named fantastic_analysis_plugin is located inside your package in a file named plugins.py.

So now, let us have a look at what do you have to have inside this file:

Listing 11 Example of plugins.py
 1"""
 2This is the module that will be exposed via the entry point declaration.
 3
 4Make sure to have all processors that you need to export in the list.
 5"""
 6
 7import mafw
 8from mafw.processor import Processor
 9from fantastic_analysis import my_processor_lib
10
11@mafw.mafw_hookimpl
12def register_processors() -> list[mafw.processor.Processor]:
13    return [my_processor_lib.Processor1, my_processor_lib.Processor2]

In this python file you need to import mafw in order to have access to the mafw_hookimpl decorator. This is a marker for pluggy so that it knows that this function should return something for the host application. The second import statement is needed only for the typing of the return value. The third one is to import your library with the processors. It can be one or as many as you have.

Inside the returned list, just put all the processors you want to export and the trick is done!

Now install in development mode your package and MAFw will be able to access all your processor library, without you having to do anything else!

What’s next

If you are still reading this tutorial, it means that you are aware of the benefits that MAFw can bring to your daily analysis task. And the best part is, there is even more to discover! On the following section, you will discover how to efficiently manage your complex analytical process by simply listing all the sequential steps in a steering file.