feat: add the ability to load plugins in config
This commit is contained in:
Mike Hennessy
parent
ac5793ceb5
commit
6dd426d71d
|
|
@ -1,5 +1,4 @@
|
|||
Contributing
|
||||
============
|
||||
# Contributing
|
||||
|
||||
All kinds of contributions to Dotbot are greatly appreciated. For someone
|
||||
unfamiliar with the code base, the most efficient way to contribute is usually
|
||||
|
|
@ -7,21 +6,19 @@ to submit a [feature request](#feature-requests) or [bug report](#bug-reports).
|
|||
If you want to dive into the source code, you can submit a [patch](#patches) as
|
||||
well, either working on your own ideas or [existing issues][issues].
|
||||
|
||||
Feature Requests
|
||||
----------------
|
||||
## Feature Requests
|
||||
|
||||
Do you have an idea for an awesome new feature for Dotbot? Please [submit a
|
||||
feature request][issue]. It's great to hear about new ideas.
|
||||
|
||||
If you are inclined to do so, you're welcome to [fork][fork] Dotbot, work on
|
||||
implementing the feature yourself, and submit a patch. In this case, it's
|
||||
*highly recommended* that you first [open an issue][issue] describing your
|
||||
_highly recommended_ that you first [open an issue][issue] describing your
|
||||
enhancement to get early feedback on the new feature that you are implementing.
|
||||
This will help avoid wasted efforts and ensure that your work is incorporated
|
||||
into the code base.
|
||||
|
||||
Bug Reports
|
||||
-----------
|
||||
## Bug Reports
|
||||
|
||||
Did something go wrong with Dotbot? Sorry about that! Bug reports are greatly
|
||||
appreciated!
|
||||
|
|
@ -31,15 +28,14 @@ as Dotbot version, operating system, configuration file, error messages, and
|
|||
steps to reproduce the bug. The more details you can include, the easier it is
|
||||
to find and fix the bug.
|
||||
|
||||
Patches
|
||||
-------
|
||||
## Patches
|
||||
|
||||
Want to hack on Dotbot? Awesome!
|
||||
|
||||
If there are [open issues][issues], you're more than welcome to work on those -
|
||||
this is probably the best way to contribute to Dotbot. If you have your own
|
||||
ideas, that's great too! In that case, before working on substantial changes to
|
||||
the code base, it is *highly recommended* that you first [open an issue][issue]
|
||||
the code base, it is _highly recommended_ that you first [open an issue][issue]
|
||||
describing what you intend to work on.
|
||||
|
||||
**Patches are generally submitted as pull requests.** Patches are also
|
||||
|
|
@ -50,6 +46,45 @@ used in the rest of the project. The version history should be clean, and
|
|||
commit messages should be descriptive and [properly
|
||||
formatted][commit-messages].
|
||||
|
||||
When preparing a patch, it's recommended that you add unit tests
|
||||
that demonstrate the bug is fixed (or that the feature works).
|
||||
You can run the tests on your local machine by installing the `dev` extras.
|
||||
The steps below do this using a virtual environment:
|
||||
|
||||
```shell
|
||||
# Create a local virtual environment
|
||||
$ python -m venv .venv
|
||||
|
||||
# Activate the virtual environment
|
||||
# Cygwin, Linux, and MacOS:
|
||||
$ . .venv/bin/activate
|
||||
# Windows Powershell:
|
||||
$ & .venv\Scripts\Activate.ps1
|
||||
|
||||
# Update pip and setuptools
|
||||
(.venv) $ python -m pip install -U pip setuptools
|
||||
|
||||
# Install dotbot and its development dependencies
|
||||
(.venv) $ python -m pip install -e .[dev]
|
||||
|
||||
# Run the unit tests
|
||||
(.venv) $ tox
|
||||
```
|
||||
|
||||
If you prefer to run the tests in an isolated container using Docker, you can
|
||||
do so with the following:
|
||||
|
||||
```
|
||||
docker run -it --rm -v "${PWD}:/dotbot" -w /dotbot python:3.10-alpine /bin/sh
|
||||
```
|
||||
|
||||
If the machine you are running Docker on has SELinux in the enforcing state, you
|
||||
will have to disable that on the container. This can be done by adding
|
||||
`--security-opt label:disable` to the above command.
|
||||
|
||||
After spawning the container, follow the same instructions as above (create a
|
||||
virtualenv, ..., run the tests).
|
||||
|
||||
---
|
||||
|
||||
If you have any questions about anything, feel free to [ask][email]!
|
||||
|
|
|
|||
23
README.md
23
README.md
|
|
@ -424,11 +424,26 @@ should do something and return whether or not it completed successfully.
|
|||
All built-in Dotbot directives are written as plugins that are loaded by
|
||||
default, so those can be used as a reference when writing custom plugins.
|
||||
|
||||
Plugins are loaded using the `--plugin` and `--plugin-dir` options, using
|
||||
either absolute paths or paths relative to the base directory. It is
|
||||
recommended that these options are added directly to the `install` script.
|
||||
See [here](plugins) for a current list of plugins.
|
||||
|
||||
See [here][plugins] for a current list of plugins.
|
||||
#### Format
|
||||
|
||||
Plugins can be loaded either by the command-line arguments `--plugin` or
|
||||
`--plugin-dir` or by the `plugins` directive. Each of these take either
|
||||
absolute paths or paths relative to the base directory.
|
||||
|
||||
When using command-line arguments to load multiple plugins you must add
|
||||
one argument for each plugin to be loaded. It is recommended to place
|
||||
these command-line arguments directly in the `install` script.
|
||||
|
||||
The `plugins` config directive is specified as an array of paths to load.
|
||||
|
||||
#### Example
|
||||
|
||||
```yaml
|
||||
- plugins:
|
||||
- dotbot-plugins/dotbot-template
|
||||
```
|
||||
|
||||
## Command-line Arguments
|
||||
|
||||
|
|
|
|||
|
|
@ -84,7 +84,7 @@ def main():
|
|||
|
||||
plugin_directories = list(options.plugin_dirs)
|
||||
if not options.disable_built_in_plugins:
|
||||
from .plugins import Clean, Create, Link, Shell
|
||||
from .plugins import Clean, Create, Link, Shell, Plugins
|
||||
plugin_paths = []
|
||||
for directory in plugin_directories:
|
||||
for plugin_path in glob.glob(os.path.join(directory, '*.py')):
|
||||
|
|
|
|||
|
|
@ -1,12 +1,21 @@
|
|||
import os
|
||||
from argparse import Namespace
|
||||
|
||||
from .context import Context
|
||||
from .plugin import Plugin
|
||||
from .messenger import Messenger
|
||||
from .context import Context
|
||||
from .plugin import Plugin
|
||||
|
||||
|
||||
class Dispatcher(object):
|
||||
def __init__(self, base_directory, only=None, skip=None, exit_on_failure=False,
|
||||
options=Namespace()):
|
||||
def __init__(
|
||||
self,
|
||||
base_directory,
|
||||
only=None,
|
||||
skip=None,
|
||||
exit_on_failure=False,
|
||||
options=Namespace(),
|
||||
):
|
||||
self._log = Messenger()
|
||||
self._setup_context(base_directory, options)
|
||||
self._load_plugins()
|
||||
|
|
@ -15,55 +24,73 @@ class Dispatcher(object):
|
|||
self._exit = exit_on_failure
|
||||
|
||||
def _setup_context(self, base_directory, options):
|
||||
path = os.path.abspath(
|
||||
os.path.expanduser(base_directory))
|
||||
path = os.path.abspath(os.path.expanduser(base_directory))
|
||||
if not os.path.exists(path):
|
||||
raise DispatchError('Nonexistent base directory')
|
||||
raise DispatchError("Nonexistent base directory")
|
||||
self._context = Context(path, options)
|
||||
|
||||
def dispatch(self, tasks):
|
||||
success = True
|
||||
for task in tasks:
|
||||
for action in task:
|
||||
if (self._only is not None and action not in self._only \
|
||||
or self._skip is not None and action in self._skip) \
|
||||
and action != 'defaults':
|
||||
self._log.info('Skipping action %s' % action)
|
||||
if (
|
||||
self._only is not None
|
||||
and action not in self._only
|
||||
or self._skip is not None
|
||||
and action in self._skip
|
||||
) and action != "defaults":
|
||||
self._log.info("Skipping action %s" % action)
|
||||
continue
|
||||
|
||||
handled = False
|
||||
if action == 'defaults':
|
||||
self._context.set_defaults(task[action]) # replace, not update
|
||||
if action == "defaults":
|
||||
self._context.set_defaults(task[action]) # replace, not update
|
||||
handled = True
|
||||
# keep going, let other plugins handle this if they want
|
||||
|
||||
for plugin in self._plugins:
|
||||
if plugin.can_handle(action):
|
||||
try:
|
||||
local_success = plugin.handle(action, task[action])
|
||||
if not local_success and self._exit:
|
||||
# The action has failed exit
|
||||
self._log.error('Action %s failed' % action)
|
||||
self._log.error("Action %s failed" % action)
|
||||
return False
|
||||
success &= local_success
|
||||
handled = True
|
||||
except Exception as err:
|
||||
self._log.error(
|
||||
'An error was encountered while executing action %s' %
|
||||
action)
|
||||
"An error was encountered while executing action %s"
|
||||
% action
|
||||
)
|
||||
self._log.debug(err)
|
||||
if self._exit:
|
||||
# There was an execption exit
|
||||
# There was an exception exit
|
||||
return False
|
||||
|
||||
if not handled:
|
||||
success = False
|
||||
self._log.error('Action %s not handled' % action)
|
||||
self._log.error("Action %s not handled" % action)
|
||||
if self._exit:
|
||||
# Invalid action exit
|
||||
return False
|
||||
|
||||
if action == "plugins":
|
||||
# Create a list of loaded plugin names
|
||||
loaded_plugins = [
|
||||
plugin.__class__.__name__ for plugin in self._plugins
|
||||
]
|
||||
|
||||
# Load plugins that haven't been loaded yet
|
||||
for plugin in Plugin.__subclasses__():
|
||||
if plugin.__name__ not in loaded_plugins:
|
||||
self._plugins.append(plugin(self._context))
|
||||
|
||||
return success
|
||||
|
||||
def _load_plugins(self):
|
||||
self._plugins = [plugin(self._context)
|
||||
for plugin in Plugin.__subclasses__()]
|
||||
self._plugins = [plugin(self._context) for plugin in Plugin.__subclasses__()]
|
||||
|
||||
|
||||
class DispatchError(Exception):
|
||||
pass
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
from .clean import Clean
|
||||
from .create import Create
|
||||
from .link import Link
|
||||
from .plugins import Plugins
|
||||
from .shell import Shell
|
||||
|
|
|
|||
|
|
@ -0,0 +1,46 @@
|
|||
import glob
|
||||
import os
|
||||
|
||||
from ..plugin import Plugin
|
||||
from ..util import module
|
||||
|
||||
|
||||
class Plugins(Plugin):
|
||||
"""
|
||||
Load plugins from a list of paths.
|
||||
"""
|
||||
|
||||
_directive = "plugins"
|
||||
_has_shown_override_message = False
|
||||
|
||||
def can_handle(self, directive):
|
||||
return directive == self._directive
|
||||
|
||||
def handle(self, directive, data):
|
||||
if directive != self._directive:
|
||||
raise ValueError("plugins cannot handle directive %s" % directive)
|
||||
return self._process_plugins(data)
|
||||
|
||||
def _process_plugins(self, data):
|
||||
success = True
|
||||
plugin_paths = []
|
||||
for item in data:
|
||||
self._log.lowinfo("Loading plugin from %s" % item)
|
||||
|
||||
plugin_path_globs = glob.glob(os.path.join(item, "*.py"))
|
||||
if not plugin_path_globs:
|
||||
success = False
|
||||
self._log.warning("Failed to load plugin from %s" % item)
|
||||
else:
|
||||
for plugin_path in plugin_path_globs:
|
||||
plugin_paths.append(plugin_path)
|
||||
|
||||
for plugin_path in plugin_paths:
|
||||
abspath = os.path.abspath(plugin_path)
|
||||
module.load(abspath)
|
||||
|
||||
if success:
|
||||
self._log.info("All commands have been executed")
|
||||
else:
|
||||
self._log.error("Some commands were not successfully executed")
|
||||
return success
|
||||
|
|
@ -4,26 +4,26 @@ import sys, os.path
|
|||
loaded_modules = []
|
||||
|
||||
def load(path):
|
||||
basename = os.path.basename(path)
|
||||
module_name, extension = os.path.splitext(basename)
|
||||
plugin = load_module(module_name, path)
|
||||
loaded_modules.append(plugin)
|
||||
basename = os.path.basename(path)
|
||||
module_name, extension = os.path.splitext(basename)
|
||||
plugin = load_module(module_name, path)
|
||||
loaded_modules.append(plugin)
|
||||
|
||||
if sys.version_info >= (3, 5):
|
||||
import importlib.util
|
||||
import importlib.util
|
||||
|
||||
def load_module(module_name, path):
|
||||
spec = importlib.util.spec_from_file_location(module_name, path)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
def load_module(module_name, path):
|
||||
spec = importlib.util.spec_from_file_location(module_name, path)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
elif sys.version_info >= (3, 3):
|
||||
from importlib.machinery import SourceFileLoader
|
||||
from importlib.machinery import SourceFileLoader
|
||||
|
||||
def load_module(module_name, path):
|
||||
return SourceFileLoader(module_name, path).load_module()
|
||||
def load_module(module_name, path):
|
||||
return SourceFileLoader(module_name, path).load_module()
|
||||
else:
|
||||
import imp
|
||||
import imp
|
||||
|
||||
def load_module(module_name, path):
|
||||
return imp.load_source(module_name, path)
|
||||
def load_module(module_name, path):
|
||||
return imp.load_source(module_name, path)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,19 @@
|
|||
"""Test that a plugin can be loaded by config file.
|
||||
|
||||
This file is copied to a location with the name "config_file.py",
|
||||
and is then loaded from within the `test_cli.py` code.
|
||||
"""
|
||||
|
||||
import os.path
|
||||
|
||||
import dotbot
|
||||
|
||||
|
||||
class ConfigFile(dotbot.Plugin):
|
||||
def can_handle(self, directive):
|
||||
return directive == "plugin_config_file"
|
||||
|
||||
def handle(self, directive, data):
|
||||
with open(os.path.abspath(os.path.expanduser("~/flag")), "w") as file:
|
||||
file.write("config file plugin loading works")
|
||||
return True
|
||||
Loading…
Reference in New Issue