Prepare for version 3.3.0

* Support nested ifs in default template (#436)
 * Support include and ifs in default template includes (#406)
 * Support environment variables in ifs in default template (#488)
 * Support != in default template (#358, #477)
 * Fix multiple classes in default template on macOS (#437)

.flake8

@@ -0,0 +1,2 @@
+[flake8]
+max-line-length = 120

.github/CONTRIBUTING.md

@@ -1,7 +1,7 @@
 # Introduction
 
-Thank you for considering contributing to **yadm**. I develop this project in my
-limited spare time, so help is very appreciated.
+Thank you for considering contributing to **yadm**. We develop this project in
+our limited spare time, so help is very appreciated.
 
 All contributors must follow our [Code of Conduct][conduct]. Please make sure
 you are welcoming and friendly during your interactions, and report any
@@ -14,9 +14,11 @@ areas.
 
 To contribute, you can:
 
+* Star the yadm repo, the star count helps others discover yadm.
 * Report [bugs](#reporting-a-bug)
 * Request [features/enhancements](#suggesting-a-feature-or-enhancement)
-* Contribute changes to [code, tests](#contributing-code), and [documentation](#improving-documentation)
+* Contribute changes to [code, tests](#contributing-code), and
+  [documentation](#improving-documentation)
 * Maintain installation [packages](#maintaining-packages)
 * Help other users by [answering support questions](#answering-support-questions)
 
@@ -69,14 +71,14 @@ Consider trying to reproduce the bug inside a docker container using the
 [yadm/testbed][] docker image. Doing so will greatly increase the likelihood of
 the problem being fixed.
 
-The easiest way to start this container, is to clone the [TheLocehiliosan/yadm
+The easiest way to start this container, is to clone the [yadm
 repo][yadm-repo], and use the `scripthost` make target. _(You will need `make`
 and `docker` installed.)_
 
 For example:
 
 ```text
-$ git clone https://github.com/TheLocehiliosan/yadm.git
+$ git clone https://github.com/yadm-dev/yadm.git
 $ cd yadm
 $ make scripthost version=1.12.0
 Starting scripthost version="1.12.0" (recording script)
@@ -202,15 +204,20 @@ these principles when making changes.
 3. Add the official repository (`upstream`) as a remote repository.
 
     ```text
-    $ git remote add upstream https://github.com/TheLocehiliosan/yadm.git
+    $ git remote add upstream https://github.com/yadm-dev/yadm.git
     ```
 
 4. Verify you can run the test harness. _(This will require dependencies:
-   `make`, `docker`, and `docker-compose`)_.
+   `make` and `docker`)_.
 
     ```text
     $ make test
     ```
+    If you don't use `docker` but an OCI engine akin to `podman`, you can set it through the `OCI` switch for every target
+
+    ```text
+    $ make test OCI=podman
+    ```
 
 5. Create a feature branch, based off the `develop` branch.
 
@@ -355,25 +362,24 @@ see if you can help.
 [attach-help]: https://help.github.com/en/articles/file-attachments-on-issues-and-pull-requests
 [commit-style]: https://chris.beams.io/posts/git-commit/#seven-rules
 [conduct]: CODE_OF_CONDUCT.md
-[contrib-hooks]: https://github.com/TheLocehiliosan/yadm/tree/master/contrib/hooks
+[contrib-hooks]: https://github.com/yadm-dev/yadm/tree/master/contrib/hooks
 [flake8]: https://pypi.org/project/flake8/
 [groff-man]: https://www.gnu.org/software/groff/manual/html_node/man.html
-[hooks-help]: https://github.com/TheLocehiliosan/yadm/blob/master/yadm.md#hooks
+[hooks-help]: https://github.com/yadm-dev/yadm/blob/master/yadm.md#hooks
 [html-proofer]: https://github.com/gjtorikian/html-proofer
 [jekyll]: https://jekyllrb.com
-[new-bug]: https://github.com/TheLocehiliosan/yadm/issues/new?template=BUG_REPORT.md
-[new-feature]: https://github.com/TheLocehiliosan/yadm/issues/new?template=FEATURE_REQUEST.md
-[open-issues]: https://github.com/TheLocehiliosan/yadm/issues
+[new-bug]: https://github.com/yadm-dev/yadm/issues/new?template=BUG_REPORT.md
+[new-feature]: https://github.com/yadm-dev/yadm/issues/new?template=FEATURE_REQUEST.md
+[open-issues]: https://github.com/yadm-dev/yadm/issues
 [pr-help]: https://help.github.com/en/articles/creating-a-pull-request-from-a-fork
 [pylint]: https://pylint.org/
 [pytest]: https://pytest.org/
-[questions]: https://github.com/TheLocehiliosan/yadm/labels/question
-[refactor]: https://github.com/TheLocehiliosan/yadm/issues/146
+[questions]: https://github.com/yadm-dev/yadm/labels/question
 [shellcheck]: https://www.shellcheck.net
 [signing-commits]: https://help.github.com/en/articles/signing-commits
 [tpope-style]: https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html
-[yadm-man]: https://github.com/TheLocehiliosan/yadm/blob/master/yadm.md
-[yadm-repo]: https://github.com/TheLocehiliosan/yadm
+[yadm-man]: https://github.com/yadm-dev/yadm/blob/master/yadm.md
+[yadm-repo]: https://github.com/yadm-dev/yadm
 [yadm/jekyll]: https://hub.docker.com/r/yadm/jekyll
 [yadm/testbed]: https://hub.docker.com/r/yadm/testbed
 [yamllint]: https://github.com/adrienverge/yamllint

.github/ISSUE_TEMPLATE.md

@@ -1,6 +1,6 @@
 <!--
 Before submitting, please search open and closed issues at
-https://github.com/TheLocehiliosan/yadm/issues to avoid duplication.
+https://github.com/yadm-dev/yadm/issues to avoid duplication.
 
 If you have found a security vulnerability, do NOT open an issue.
 Email yadm@yadm.io instead.

.github/ISSUE_TEMPLATE/BUG_REPORT.md

@@ -8,7 +8,7 @@ assignees: ''
 ---
 <!--
 Before submitting, please search open and closed issues at
-https://github.com/TheLocehiliosan/yadm/issues to avoid duplication.
+https://github.com/yadm-dev/yadm/issues to avoid duplication.
 
 If you have found a security vulnerability, do NOT open an issue.
 Email yadm@yadm.io instead.
@@ -25,10 +25,10 @@ Can this be reproduced with the yadm/testbed docker image: [Yes/No]
 Consider trying to reproduce the bug inside a docker container using the
 yadm/testbed docker image. https://hub.docker.com/r/yadm/testbed
 
-The easiest way to start this container, is to clone the TheLocehiliosan/yadm
+The easiest way to start this container, is to clone the yadm-dev/yadm
 repo, and use the "scripthost" make target. For example:
 
-  $ git clone https://github.com/TheLocehiliosan/yadm.git
+  $ git clone https://github.com/yadm-dev/yadm.git
   $ cd yadm
   $ make scripthost version=1.11.0
   Starting scripthost version="1.11.0" (recording script)

.github/ISSUE_TEMPLATE/FEATURE_REQUEST.md

@@ -8,7 +8,7 @@ assignees: ''
 ---
 <!--
 Before submitting, please search open and closed issues at
-https://github.com/TheLocehiliosan/yadm/issues to avoid duplication.
+https://github.com/yadm-dev/yadm/issues to avoid duplication.
 -->
 
 ### Is your feature request related to a problem? Please describe.

.github/ISSUE_TEMPLATE/OTHER.md

@@ -8,7 +8,7 @@ assignees: ''
 ---
 <!--
 Before submitting, please search open and closed issues at
-https://github.com/TheLocehiliosan/yadm/issues to avoid duplication.
+https://github.com/yadm-dev/yadm/issues to avoid duplication.
 -->
 
 ### This issue is about

.github/ISSUE_TEMPLATE/SUPPORT.md

@@ -8,11 +8,11 @@ assignees: ''
 ---
 <!--
 Before submitting, please search open and closed issues at
-https://github.com/TheLocehiliosan/yadm/issues to avoid duplication.
+https://github.com/yadm-dev/yadm/issues to avoid duplication.
 
 Please also consult:
   * The FAQ: https://yadm.io/docs/faq
-  * The manual: https://github.com/TheLocehiliosan/yadm/blob/master/yadm.md
+  * The manual: https://github.com/yadm-dev/yadm/blob/master/yadm.md
 -->
 
 ### This question is about

.github/PULL_REQUEST_TEMPLATE.md

@@ -29,6 +29,6 @@ Be sure to preface the issue/PR numbers with a "#".
 
 Please review [yadm's Contributing Guide][3] for best practices.
 
-[1]: https://github.com/TheLocehiliosan/yadm/blob/master/.github/CONTRIBUTING.md#test-conventions
+[1]: https://github.com/yadm-dev/yadm/blob/master/.github/CONTRIBUTING.md#test-conventions
 [2]: https://help.github.com/en/articles/signing-commits
-[3]: https://github.com/TheLocehiliosan/yadm/blob/master/.github/CONTRIBUTING.md
+[3]: https://github.com/yadm-dev/yadm/blob/master/.github/CONTRIBUTING.md

.github/workflows/schedule.yml

@@ -0,0 +1,20 @@
+---
+name: Scheduled Site Tests
+on:  # yamllint disable-line rule:truthy
+  schedule:
+    - cron: "0 0 1 * *"  # Monthly
+jobs:
+  Tests:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+        with:
+          ref: gh-pages
+      - run: >-
+          docker create -t
+          --name yadm-website
+          --entrypoint test/validate
+          yadm/jekyll:2024-10-31;
+          docker cp ./ yadm-website:/srv/jekyll
+      - name: Test Site
+        run: docker start yadm-website -a

.github/workflows/stale.yml

@@ -0,0 +1,25 @@
+---
+name: Close Stale Issues
+on:  # yamllint disable-line rule:truthy
+  schedule:
+    - cron: "30 1 * * *"  # Daily
+jobs:
+  Stale:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/stale@v4
+        with:
+          close-issue-message: >-
+            This issue was closed because it has been labeled as stale for 7
+            days with no activity.
+          days-before-close: 7
+          days-before-stale: 60
+          exempt-all-assignees: true
+          exempt-issue-labels: in develop, 1, 2, 3
+          exempt-pr-labels: in develop, 1, 2, 3
+          stale-issue-label: stale
+          stale-issue-message: >-
+            This issue has been labeled as stale because it has been open 60
+            days with no activity. Remove stale label or comment or this will
+            be closed in 7 days.
+          stale-pr-label: stale

.github/workflows/test.yml

@@ -0,0 +1,13 @@
+---
+name: Tests
+on:  # yamllint disable-line rule:truthy
+  - push
+  - pull_request
+  - workflow_dispatch
+jobs:
+  Tests:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Tests
+        run: make test

.gitignore

@@ -2,5 +2,6 @@
 .jekyll-metadata
 .pytest_cache
 .sass-cache
+.testyadm
 _site
 testenv

.travis.yml

@@ -1,8 +0,0 @@
----
-language: minimal
-services:
-  - docker
-before_install:
-  - docker pull yadm/testbed:2019-09-25
-script:
-  - docker run -t --rm -v "$PWD:/yadm:ro" yadm/testbed:2019-09-25

CHANGES

@@ -1,3 +1,105 @@
+3.3.0
+  * Support nested ifs in default template (#436)
+  * Support include and ifs in default template includes (#406)
+  * Support environment variables in ifs in default template (#488)
+  * Support != in default template (#358, #477)
+  * Fix multiple classes in default template on macOS (#437)
+
+3.2.2
+  * Support spaces in distro/distro-family (#432)
+  * Fix zsh hanging when tab completing add/checkout (#417)
+  * Add yadm-untracked script to contributed files (#418)
+  * Fix documentation typos (#425)
+  * Support docker-like OCI engines for dev testing (#431)
+
+3.2.1
+  * Fix Bash 3 bad array subscript bug (#411)
+
+3.2.0
+  * Support architecture for alternates/templates (#202, #203, #393)
+  * Support distro_family for alternates/templates (#213)
+  * Support setting multiple classes (#185, #304)
+  * Support environment variables in default template processor (#347)
+  * Update version command to include Bash & Git versions (#377)
+
+3.1.1
+  * Fix clone support for older versions of Git (#348)
+  * Fix support for multiple GPG recipients (#342)
+  * Find symlinks in bootstrap-in-dir (#340)
+
+3.1.0
+  * Use `git clone` directly during clone (#289, #323)
+  * Fix compatibility bug with Git completions (#318, #321)
+  * Support relative paths for --yadm-* and -w (#301)
+  * Improve parsing of if-statement in default template (#303)
+  * Read files without running cat in subshells (#317)
+  * Improve portability of updating read-only files (#320)
+  * Various code improvements (#306, #307, #311)
+
+3.0.2
+  * Fix parsing by sh (#299)
+
+3.0.1
+  * Improve handling of submodules at upgrade (#284, #285, #293)
+  * Improve Zsh completions (#292, #298)
+  * Use stderr for error messages (#297)
+
+3.0.0
+  * Support encryption with OpenSSL (#138)
+  * Support "include" directive in built-in template processor (#255)
+  * Support extensions for alternate files and templates (#257)
+  * Improve support for default branches (#231, #232)
+  * Add --version and --help as yadm internal commands (#267)
+  * Improve support for XDG base directory specification
+  * Use XDG_DATA_HOME used for encrypted data and repository (#208)
+  * Default repo is now ~/.local/share/yadm/repo.git
+  * Default encrypted archive is now ~/.local/share/yadm/archive
+  * Improve shell completions (#238, #274, #275)
+  * Remove support for YADM_COMPATIBILITY=1 (#242)
+  * Remove deprecated option cygwin-copy
+  * Fix template mode inheritance on FreeBSD (#243, #246)
+  * Fix hook execution under MinGW (#150)
+  * Improve compatibility with Oil shell (#210)
+
+2.5.0
+  * Support for transcrypt (#197)
+  * Support ESH templates (#220)
+  * Preserve file mode of template (#193)
+  * Fish shell completions (#224)
+  * Fix alt processing when worktree is `/` (#198)
+  * Assert config directory if missing (#226, #227)
+  * Documentation improvements (#229)
+
+2.4.0
+  * Support multiple keys in `yadm.gpg-recipient` (#139)
+  * Ensure all templates are written atomically (#142)
+  * Add encrypt_with_checksums to the hooks collection (#188)
+  * Escape white space in YADM_HOOK_FULL_COMMAND (#187)
+  * Improve parsing of os-release (#194)
+  * Improve identification of WSL (#196)
+  * Fix troff warnings emitted by man page (#195)
+  * Write encrypt-based exclusions during decrypt
+
+2.3.0
+  * Support git-crypt (#168)
+  * Support specifying a command after `yadm enter`
+  * Expose GIT_WORK_TREE during `yadm enter` (#160)
+  * Support GNUPGHOME environment variable (#134)
+  * Assert private dirs, only when worktree = $HOME (#171)
+
+2.2.0
+  * Resolve hostname using `uname -n` (#182)
+  * Use /etc/os-release if lsb_release is missing (#175)
+  * Issue warning for any invalid alternates found (#183)
+  * Add support for gawk (#180)
+
+2.1.0
+  * Use relative symlinks for alternates (#100, #177)
+  * Support double-star globs in .config/yadm/encrypt (#109)
+  * Improve bash completion (#136)
+  * Update docs about using magit (#123)
+  * Note exception for WSL (#113)
+
 2.0.1
   * Fix bug with worktree permissions (#174)
 

CONTRIBUTORS

@@ -1,21 +1,54 @@
 CONTRIBUTORS
 
 Tim Byrne
+Erik Flodin
+Martin Zuther
+Jan Schulz
+Ross Smith II
+Jonathan Daigle
+Luis López
+Tin Lai
 Espen Henriksen
 Cameron Eagans
-Ross Smith II
 Klas Mellbourn
-Jan Schulz
-Satoshi Ohki
-Siôn Le Roux
-Sébastien Gross
-Thomas Luzat
+James Clark
+Glenn Waters
+Nicolas signed-log FORMICHELLA
 Tomas Cernaj
-Uroš Golja
-Brayden Banks
-japm48
-Daniel Wagenknecht
-Franciszek Madej
-Mateusz Piotrowski
+Joshua Cold
+jonasc
+Nicolas stig124 FORMICHELLA
+Chad Wade Day, Jr
+Sébastien Gross
+David Mandelberg
+Paulo Köch
+Oren Zipori
+Daniel Gray
 Paraplegic Racehorse
+Siôn Le Roux
+Mateusz Piotrowski
+japm48
+Uroš Golja
+Satoshi Ohki
+Jonas
+Franciszek Madej
+Daniel Wagenknecht
+Stig Palmquist
 Patrick Hof
+Samisafool
+LFdev
+con-f-use
+Bram Ceulemans
+Travis A. Everett
+Sheng Yang
+Jared Smartt
+Adam Jimerson
+Tim Condit
+Thomas Luzat
+Russ Allbery
+Patrick Roddy
+dessert1
+Brayden Banks
+Alexandre GV
+addshore
+Felipe S. S. Schneider

Dockerfile

@@ -1,52 +0,0 @@
-FROM ubuntu:18.04
-MAINTAINER Tim Byrne <sultan@locehilios.com>
-
-# No input during build
-ENV DEBIAN_FRONTEND noninteractive
-
-# UTF8 locale
-RUN apt-get update && apt-get install -y locales
-RUN locale-gen en_US.UTF-8
-ENV LANG='en_US.UTF-8' LANGUAGE='en_US:en' LC_ALL='en_US.UTF-8'
-
-# Convenience settings for the testbed's root account
-RUN echo 'set -o vi' >> /root/.bashrc
-
-# Install prerequisites
-RUN \
-  apt-get update && \
-  apt-get install -y \
-    curl \
-    expect \
-    git \
-    gnupg1 \
-    lsb-release \
-    make \
-    python3-pip \
-    shellcheck=0.4.6-1 \
-    vim \
-  ;
-RUN pip3 install \
-      envtpl \
-      j2cli \
-      flake8==3.7.8 \
-      pylint==2.4.1 \
-      pytest==5.1.3 \
-      yamllint==1.17.0 \
-    ;
-
-# Force GNUPG version 1 at path /usr/bin/gpg
-RUN ln -fs /usr/bin/gpg1 /usr/bin/gpg
-
-# Create a flag to identify when running inside the yadm testbed
-RUN touch /.yadmtestbed
-
-# /yadm will be the work directory for all tests
-# docker commands should mount the local yadm project as /yadm
-WORKDIR /yadm
-
-# Create a Makefile to be used if no /yadm volume is mounted
-RUN echo "test:\n\t@echo 'The yadm project must be mounted at /yadm'\n\t@echo 'Try using a docker parameter like -v \"\$\$PWD:/yadm:ro\"'\n\t@false" > /yadm/Makefile
-
-# By default, run all tests defined
-CMD make test

Makefile

@@ -1,4 +1,6 @@
 PYTESTS = $(wildcard test/test_*.py)
+IMAGE = docker.io/yadm/testbed:2023-07-12
+OCI = docker
 
 .PHONY: all
 all:
@@ -22,7 +24,7 @@ usage:
 	@echo
 	@echo '  make testhost [version=VERSION]'
 	@echo '    - Create an ephemeral container for doing adhoc yadm testing. The'
-	@echo '      HEAD revision of yadm will be used unless "version" is'
+	@echo '      working copy version of yadm will be used unless "version" is'
 	@echo '      specified. "version" can be set to any commit, branch, tag, etc.'
 	@echo '      The targeted "version" will be retrieved from the repo, and'
 	@echo '      linked into the container as a local volume.'
@@ -32,7 +34,7 @@ usage:
 	@echo '      exiting the shell, a log of the commands used to illustrate the'
 	@echo '      problem will be written to the file "script.txt". This file can'
 	@echo '      be useful to developers to make a repeatable test for the'
-	@echo '      problem.'
+	@echo '      problem. The version parameter works as for "testhost" above.'
 	@echo
 	@echo 'LINTING'
 	@echo
@@ -81,7 +83,7 @@ usage:
 # Make it possible to run make specifying a py.test test file
 .PHONY: $(PYTESTS)
 $(PYTESTS):
-	@$(MAKE) test testargs="-k $@ $(testargs)"
+	@$(MAKE) test testargs="$@ $(testargs)"
 %.py:
 	@$(MAKE) test testargs="-k $@ $(testargs)"
 
@@ -92,65 +94,75 @@ test:
 		cd /yadm && \
 		py.test -v $(testargs); \
 	else \
-		if command -v "docker-compose" >/dev/null 2>&1; then \
-			docker-compose run --rm testbed make test testargs="$(testargs)"; \
-		else \
-			echo "Sorry, this make test requires docker-compose to be installed."; \
-			false; \
-		fi \
+		$(MAKE) -s require-docker && \
+		$(OCI) run \
+			--rm -t$(shell test -t 0 && echo i) \
+			-v "$(CURDIR):/yadm:ro" \
+			$(IMAGE) \
+			make test testargs="$(testargs)"; \
 	fi
 
+.PHONY: .testyadm
+.testyadm: version ?= local
+.testyadm:
+	@rm -f $@
+	@if [ "$(version)" = "local" ]; then \
+		ln -sf yadm $@; \
+		echo "Using local yadm ($$(git describe --tags --dirty))"; \
+	else \
+		git show $(version):yadm > $@; \
+		echo "Using yadm version $$(git describe --tags $(version))"; \
+	fi
+	@chmod a+x $@
+
 .PHONY: testhost
-testhost: require-docker
-	@version=HEAD
-	@rm -rf /tmp/testhost
-	@git show $(version):yadm > /tmp/testhost
-	@chmod a+x /tmp/testhost
-	@echo Starting testhost version=\"$$version\"
-	@docker run \
+testhost: require-docker .testyadm
+	@echo "Starting testhost"
+	@$(OCI) run \
 		-w /root \
 		--hostname testhost \
 		--rm -it \
-		-v "/tmp/testhost:/bin/yadm:ro" \
-		yadm/testbed:2019-09-25 \
+		-v "$(CURDIR)/.testyadm:/bin/yadm:ro" \
+		$(IMAGE) \
 		bash -l
 
 .PHONY: scripthost
-scripthost: require-docker
-	@version=HEAD
-	@rm -rf /tmp/testhost
-	@git show $(version):yadm > /tmp/testhost
-	@chmod a+x /tmp/testhost
-	@echo Starting scripthost version=\"$$version\" \(recording script\)
+scripthost: require-docker .testyadm
+	@echo "Starting scripthost \(recording script\)"
 	@printf '' > script.gz
-	@docker run \
+	@$(OCI) run \
 		-w /root \
 		--hostname scripthost \
 		--rm -it \
-		-v "$$PWD/script.gz:/script.gz:rw" \
-		-v "/tmp/testhost:/bin/yadm:ro" \
-		yadm/testbed:2019-09-25 \
+		-v "$(CURDIR)/script.gz:/script.gz:rw" \
+		-v "$(CURDIR)/.testyadm:/bin/yadm:ro" \
+		$(IMAGE) \
 		bash -c "script /tmp/script -q -c 'bash -l'; gzip < /tmp/script > /script.gz"
 	@echo
-	@echo "Script saved to $$PWD/script.gz"
+	@echo "Script saved to $(CURDIR)/script.gz"
 
 
 .PHONY: testenv
 testenv:
 	@echo 'Creating a local virtual environment in "testenv/"'
 	@echo
-	virtualenv --python=python3 testenv
+	@rm -rf testenv
+	python3 -m venv --clear testenv
 	testenv/bin/pip3 install --upgrade pip setuptools
-	testenv/bin/pip3 install --upgrade \
-		flake8==3.7.8 \
-		pylint==2.4.1 \
-		pytest==5.1.3 \
-		yamllint==1.17.0 \
-	;
+	testenv/bin/pip3 install --upgrade -r test/requirements.txt;
+	@for v in $$(sed -En -e 's:.*/yadm-([0-9.]+)$$:\1:p' test/Dockerfile); do \
+		git show $$v:yadm > testenv/bin/yadm-$$v; \
+		chmod +x testenv/bin/yadm-$$v; \
+	done
 	@echo
 	@echo 'To activate this test environment type:'
 	@echo '  source testenv/bin/activate'
 
+.PHONY: image
+image:
+	@$(OCI) build -f test/Dockerfile . -t "$(IMAGE)"
+
+
 .PHONY: man
 man:
 	@groff -man -Tascii ./yadm.1 | less
@@ -164,21 +176,26 @@ man-ps:
 	@groff -man -Tps ./yadm.1 > yadm.ps
 
 yadm.md: yadm.1
-	@groff -man -Tascii ./yadm.1 | col -bx | sed 's/^[A-Z]/## &/g' | sed '/yadm(1)/d' > yadm.md
+	@groff -man -Tutf8 -Z ./yadm.1 | grotty -c | col -bx | sed 's/^[A-Z]/## &/g' | sed '/yadm(1)/d' > yadm.md
 
 .PHONY: contrib
+contrib: SHELL = /bin/bash
 contrib:
-	@echo "CONTRIBUTORS\n" > CONTRIBUTORS
-	@git shortlog -ns master gh-pages develop dev-pages | cut -f2 >> CONTRIBUTORS
+	@echo -e "CONTRIBUTORS\n" > CONTRIBUTORS
+	@IFS=$$'\n'; for author in $$(git shortlog -ns master gh-pages develop dev-pages | cut -f2); do \
+		git log master gh-pages develop dev-pages \
+			--author="$$author" --format=tformat: --numstat | \
+			awk "{sum += \$$1 + \$$2} END {print sum \"\t\" \"$$author\"}"; \
+	done | sort -nr | cut -f2 >> CONTRIBUTORS
 
 .PHONY: install
 install:
 	@[ -n "$(PREFIX)" ] || { echo "PREFIX is not set"; exit 1; }
 	@{\
 		set -e                                               ;\
-		bin="$(PREFIX)/bin"                                  ;\
-		doc="$(PREFIX)/share/doc/yadm"                       ;\
-		man="$(PREFIX)/share/man/man1"                       ;\
+		bin="$(DESTDIR)$(PREFIX)/bin"                        ;\
+		doc="$(DESTDIR)$(PREFIX)/share/doc/yadm"             ;\
+		man="$(DESTDIR)$(PREFIX)/share/man/man1"             ;\
 		install -d "$$bin" "$$doc" "$$man"                   ;\
 		install -m 0755 yadm "$$bin"                         ;\
 		install -m 0644 yadm.1 "$$man"                       ;\
@@ -188,11 +205,11 @@ install:
 
 .PHONY: sync-clock
 sync-clock:
-	docker run --rm --privileged alpine hwclock -s
+	$(OCI) run --rm --privileged alpine hwclock -s
 
 .PHONY: require-docker
 require-docker:
-	@if ! command -v "docker" >/dev/null 2>&1; then \
-		echo "Sorry, this make target requires docker to be installed."; \
+	@if ! command -v $(OCI) > /dev/null 2>&1; then \
+		echo "Sorry, this make target requires docker to be installed, to use another docker-compatible engine, like podman, re-run the make command adding OCI=podman"; \
 		false; \
 	fi

README.md

@@ -3,50 +3,84 @@
 [![Latest Version][releases-badge]][releases-link]
 [![Homebrew Version][homebrew-badge]][homebrew-link]
 [![OBS Version][obs-badge]][obs-link]
-[![Arch Version][aur-badge]][aur-link]
+[![Arch Version][arch-badge]][arch-link]
 [![License][license-badge]][license-link]<br />
 [![Master Update][master-date]][master-commits]
 [![Develop Update][develop-date]][develop-commits]
 [![Website Update][website-date]][website-commits]<br />
-[![Master Status][master-badge]][travis-ci]
-[![Develop Status][develop-badge]][travis-ci]
-[![GH Pages Status][gh-pages-badge]][travis-ci]
-[![Dev Pages Status][dev-pages-badge]][travis-ci]
+[![Master Status][master-badge]][workflow-master]
+[![Develop Status][develop-badge]][workflow-develop]
+[![GH Pages Status][gh-pages-badge]][workflow-gh-pages]
+[![Dev Pages Status][dev-pages-badge]][workflow-dev-pages]
 
 [https://yadm.io/][website-link]
 
-[**yadm**][website-link] is a tool for managing [dotfiles][].
+**yadm** is a tool for managing [dotfiles][].
 
 * Based on [Git][], with full range of Git's features
-* Supports system-specific alternative files
-* Encryption of private data using [GnuPG][]
+* Supports system-specific alternative files or templated files
+* Encryption of private data using [GnuPG][], [OpenSSL][], [transcrypt][], or
+  [git-crypt][]
 * Customizable initialization (bootstrapping)
+* Customizable hooks for before and after any operation
 
-Features, usage, examples and installation instructions can be found on the
-[website][website-link].
+Complete features, usage, examples and installation instructions can be found on
+the [yadm.io][website-link] website.
+
+## A very quick tour
+
+    # Initialize a new repository
+    yadm init
+
+    # Clone an existing repository
+    yadm clone <url>
+
+    # Add files/changes
+    yadm add <important file>
+    yadm commit
+
+    # Encrypt your ssh key
+    echo '.ssh/id_rsa' > ~/.config/yadm/encrypt
+    yadm encrypt
+
+    # Later, decrypt your ssh key
+    yadm decrypt
+
+    # Create different files for Linux vs MacOS
+    yadm add path/file.cfg##os.Linux
+    yadm add path/file.cfg##os.Darwin
+
+If you enjoy using yadm, consider adding a star to the repository on GitHub.
+The star count helps others discover yadm.
 
 [Git]: https://git-scm.com/
 [GnuPG]: https://gnupg.org/
-[aur-badge]: https://img.shields.io/aur/version/yadm-git.svg
-[aur-link]: https://aur.archlinux.org/packages/yadm-git
-[dev-pages-badge]: https://img.shields.io/travis/TheLocehiliosan/yadm/dev-pages.svg?label=dev-pages
-[develop-badge]: https://img.shields.io/travis/TheLocehiliosan/yadm/develop.svg?label=develop
-[develop-commits]: https://github.com/TheLocehiliosan/yadm/commits/develop
-[develop-date]: https://img.shields.io/github/last-commit/TheLocehiliosan/yadm/develop.svg?label=develop
+[OpenSSL]: https://www.openssl.org/
+[arch-badge]: https://img.shields.io/archlinux/v/extra/any/yadm
+[arch-link]: https://archlinux.org/packages/extra/any/yadm/
+[dev-pages-badge]: https://img.shields.io/github/actions/workflow/status/yadm-dev/yadm/test.yml?branch=dev-pages
+[develop-badge]: https://img.shields.io/github/actions/workflow/status/yadm-dev/yadm/test.yml?branch=develop
+[develop-commits]: https://github.com/yadm-dev/yadm/commits/develop
+[develop-date]: https://img.shields.io/github/last-commit/yadm-dev/yadm/develop.svg?label=develop
 [dotfiles]: https://en.wikipedia.org/wiki/Hidden_file_and_hidden_directory
-[gh-pages-badge]: https://img.shields.io/travis/TheLocehiliosan/yadm/gh-pages.svg?label=gh-pages
+[gh-pages-badge]: https://img.shields.io/github/actions/workflow/status/yadm-dev/yadm/test.yml?branch=gh-pages
+[git-crypt]: https://github.com/AGWA/git-crypt
 [homebrew-badge]: https://img.shields.io/homebrew/v/yadm.svg
 [homebrew-link]: https://formulae.brew.sh/formula/yadm
-[license-badge]: https://img.shields.io/github/license/TheLocehiliosan/yadm.svg
-[license-link]: https://github.com/TheLocehiliosan/yadm/blob/master/LICENSE
-[master-badge]: https://img.shields.io/travis/TheLocehiliosan/yadm/master.svg?label=master
-[master-commits]: https://github.com/TheLocehiliosan/yadm/commits/master
-[master-date]: https://img.shields.io/github/last-commit/TheLocehiliosan/yadm/master.svg?label=master
-[obs-badge]: https://img.shields.io/badge/OBS-v2.0.1-blue
+[license-badge]: https://img.shields.io/github/license/yadm-dev/yadm.svg
+[license-link]: https://github.com/yadm-dev/yadm/blob/master/LICENSE
+[master-badge]: https://img.shields.io/github/actions/workflow/status/yadm-dev/yadm/test.yml?branch=master
+[master-commits]: https://github.com/yadm-dev/yadm/commits/master
+[master-date]: https://img.shields.io/github/last-commit/yadm-dev/yadm/master.svg?label=master
+[obs-badge]: https://img.shields.io/badge/OBS-v3.3.0-blue
 [obs-link]: https://software.opensuse.org//download.html?project=home%3ATheLocehiliosan%3Ayadm&package=yadm
-[releases-badge]: https://img.shields.io/github/tag/TheLocehiliosan/yadm.svg?label=latest+release
-[releases-link]: https://github.com/TheLocehiliosan/yadm/releases
-[travis-ci]: https://travis-ci.org/TheLocehiliosan/yadm/branches
-[website-commits]: https://github.com/TheLocehiliosan/yadm/commits/gh-pages
-[website-date]: https://img.shields.io/github/last-commit/TheLocehiliosan/yadm/gh-pages.svg?label=website
+[releases-badge]: https://img.shields.io/github/tag/yadm-dev/yadm.svg?label=latest+release
+[releases-link]: https://github.com/yadm-dev/yadm/releases
+[transcrypt]: https://github.com/elasticdog/transcrypt
+[website-commits]: https://github.com/yadm-dev/yadm/commits/gh-pages
+[website-date]: https://img.shields.io/github/last-commit/yadm-dev/yadm/gh-pages.svg?label=website
 [website-link]: https://yadm.io/
+[workflow-dev-pages]: https://github.com/yadm-dev/yadm/actions?query=workflow%3a%22test+site%22+branch%3adev-pages
+[workflow-develop]: https://github.com/yadm-dev/yadm/actions?query=workflow%3ATests+branch%3Adevelop
+[workflow-gh-pages]: https://github.com/yadm-dev/yadm/actions?query=workflow%3a%22test+site%22+branch%3agh-pages
+[workflow-master]: https://github.com/yadm-dev/yadm/actions?query=workflow%3ATests+branch%3Amaster

bootstrap

@@ -29,13 +29,13 @@
 #   source <(curl -L bootstrap.yadm.io)
 #
 
-YADM_REPO="https://github.com/TheLocehiliosan/yadm"
+YADM_REPO="https://github.com/yadm-dev/yadm"
 YADM_RELEASE=${release:-master}
 REPO_URL=""
 
 function _private_yadm() {
   unset -f yadm
-  if command -v yadm >/dev/null 2>&1; then
+  if command -v yadm &> /dev/null; then
     echo "Found yadm installed locally, removing remote yadm() function"
     unset -f _private_yadm
     command yadm "$@"
@@ -57,7 +57,7 @@ function remote_yadm() {
 }
 
 function ask_about_source() {
-  if ! command -v yadm >/dev/null 2>&1; then
+  if ! command -v yadm &> /dev/null; then
     echo
     echo "***************************************************"
     echo "yadm is NOT currently installed."

completion/README.md

@@ -1,36 +1,47 @@
 # Installation
 
-## Bash completions
 ### Prerequisites
-**yadm** completion only works if Git completions are also enabled.
 
-### Homebrew
-If using `homebrew` to install **yadm**, completions should automatically be handled if you also install `brew install bash-completion`. This might require you to include the main completion script in your own bashrc file like this:
-```
+Bash and Zsh completion only works if Git completions are also enabled.
+
+## Homebrew
+
+If using `homebrew` to install yadm, Bash, Zsh, and Fish completions should
+automatically be installed. For Bash and Zsh, you also must install
+`bash-completion` or `zsh-completions`. This might require you to include the
+main completion script in your own shell configuration like this:
+
+```bash
 [ -f /usr/local/etc/bash_completion ] && source /usr/local/etc/bash_completion
 ```
 
-### Manual installation
+## Bash (manual installation)
+
 Copy the completion script locally, and add this to you bashrc:
-```
-[ -f /full/path/to/yadm.bash_completion ] && source /full/path/to/yadm.bash_completion
+
+```bash
+[ -f /path/to/yadm/completion/bash/yadm ] && source /path/to/yadm/completion/bash/yadm
 ```
 
-## Zsh completions
-### Homebrew
-If using `homebrew` to install **yadm**, completions should handled automatically.
+## Zsh (manual installation)
 
-### Manual installation
-Copy the completion script `yadm.zsh_completion` locally, rename it to `_yadm`, and add the containing folder to `$fpath` in `.zshrc`:
-```
-fpath=(/path/to/folder/containing_yadm $fpath)
+Add the `completion/zsh` folder to `$fpath` in `.zshrc`:
+
+```zsh
+fpath=(/path/to/yadm/completion/zsh $fpath)
 autoload -U compinit
 compinit
 ```
 
-### Installation using [zplug](https://github.com/b4b4r07/zplug)
+## Zsh (using [zplug](https://github.com/b4b4r07/zplug))
+
 Load `_yadm` as a plugin in your `.zshrc`:
-```
+
+```zsh
 fpath=("$ZPLUG_HOME/bin" $fpath)
-zplug "TheLocehiliosan/yadm", rename-to:_yadm, use:"completion/yadm.zsh_completion", as:command, defer:2
+zplug "yadm-dev/yadm", use:"completion/zsh/_yadm", as:command, defer:2
 ```
+
+## Fish (manual installation)
+
+Copy the completion script `yadm.fish` to any folder within `$fish_complete_path`. For example, for local installation, you can copy it to `$HOME/.config/fish/completions/` and it will be loaded when `yadm` is invoked.

completion/bash/yadm

@@ -1,10 +1,12 @@
 # test if git completion is missing, but loader exists, attempt to load
-if ! declare -F _git > /dev/null && declare -F _completion_loader > /dev/null; then
-  _completion_loader git
+if ! declare -F _git > /dev/null && ! declare -F __git_wrap__git_main > /dev/null; then
+  if declare -F _completion_loader > /dev/null; then
+    _completion_loader git
+  fi
 fi
 
 # only operate if git completion is present
-if declare -F _git > /dev/null; then
+if declare -F _git > /dev/null || declare -F __git_wrap__git_main > /dev/null; then
 
   _yadm() {
 
@@ -18,7 +20,7 @@ if declare -F _git > /dev/null; then
       antepenultimate=${COMP_WORDS[COMP_CWORD-2]}
     fi
 
-    local GIT_DIR
+    local -x GIT_DIR
     # shellcheck disable=SC2034
     GIT_DIR="$(yadm introspect repo 2>/dev/null)"
 
@@ -60,19 +62,37 @@ if declare -F _git > /dev/null; then
 		  ;;
     esac
 
+    local yadm_switches=( $(yadm introspect switches 2>/dev/null) )
+
     # this condition is so files are completed properly for --yadm-xxx options
-    if [[ ! "$penultimate" =~ ^- ]]; then
+    if [[ " ${yadm_switches[*]} " != *" $penultimate "* ]]; then
       # TODO: somehow solve the problem with [--yadm-xxx option] being
       #       incompatible with what git expects, namely [--arg=option]
-      _git
+      if declare -F _git > /dev/null; then
+        _git
+      else
+        __git_wrap__git_main
+      fi
     fi
     if [[ "$current" =~ ^- ]]; then
       local matching
-      matching=$(compgen -W "$(yadm introspect switches 2>/dev/null)" -- "$current")
+      matching=$(compgen -W "${yadm_switches[*]}" -- "$current")
       __gitcompappend "$matching"
     fi
 
-    if [ "$COMP_CWORD" == 1 ] || [[ "$antepenultimate" =~ ^- ]] ; then
+    # Find the index of where the sub-command argument should go.
+    local command_idx
+    for (( command_idx=1 ; command_idx < ${#COMP_WORDS[@]} ; command_idx++ )); do
+      local command_idx_arg="${COMP_WORDS[$command_idx]}"
+      if [[ " ${yadm_switches[*]} " = *" $command_idx_arg "* ]]; then
+        let command_idx++
+      elif [[ "$command_idx_arg" = -* ]]; then
+        :
+      else
+        break
+      fi
+    done
+    if [[ "$COMP_CWORD" = "$command_idx" ]]; then
       local matching
       matching=$(compgen -W "$(yadm introspect commands 2>/dev/null)" -- "$current")
       __gitcompappend "$matching"

completion/fish/yadm.fish

@@ -0,0 +1,77 @@
+#!/usr/bin/fish
+
+function __fish_yadm_universial_optspecs
+    string join \n 'a-yadm-dir=' 'b-yadm-repo=' 'c-yadm-config=' \
+                    'd-yadm-encrypt=' 'e-yadm-archive=' 'f-yadm-bootstrap='
+end
+
+function __fish_yadm_needs_command
+    # Figure out if the current invocation already has a command.
+    set -l cmd (commandline -opc)
+    set -e cmd[1]
+    argparse -s (__fish_yadm_universial_optspecs) -- $cmd 2>/dev/null
+    or return 0
+    if set -q argv[1]
+        echo $argv[1]
+        return 1
+    end
+    return 0
+end
+
+function __fish_yadm_using_command
+    set -l cmd (__fish_yadm_needs_command)
+    test -z "$cmd"
+    and return 1
+    contains -- $cmd $argv
+    and return 0
+end
+
+# yadm's specific autocomplete
+complete -x -c yadm -n '__fish_yadm_needs_command'       -a 'clone'      -d 'Clone an existing repository'
+complete -F -c yadm -n '__fish_yadm_using_command clone' -s w            -d 'work-tree to use (default: $HOME)'
+complete -f -c yadm -n '__fish_yadm_using_command clone' -s b            -d 'branch to clone'
+complete -x -c yadm -n '__fish_yadm_using_command clone' -s f            -d 'force to overwrite'
+complete -x -c yadm -n '__fish_yadm_using_command clone' -l bootstrap    -d 'force bootstrap to run'
+complete -x -c yadm -n '__fish_yadm_using_command clone' -l no-bootstrap -d 'prevent bootstrap from beingrun'
+
+complete -x -c yadm -n '__fish_yadm_needs_command' -a 'alt'       -d 'Create links for alternates'
+complete -x -c yadm -n '__fish_yadm_needs_command' -a 'bootstrap' -d 'Execute $HOME/.config/yadm/bootstrap'
+complete -x -c yadm -n '__fish_yadm_needs_command' -a 'perms'     -d 'Fix perms for private files'
+complete -x -c yadm -n '__fish_yadm_needs_command' -a 'enter'     -d 'Run sub-shell with GIT variables set'
+complete    -c yadm -n '__fish_yadm_needs_command' -a 'git-crypt' -d 'Run git-crypt commands for the yadm repo'
+complete -x -c yadm -n '__fish_yadm_needs_command' -a 'help'      -d 'Print a summary of yadm commands'
+complete -x -c yadm -n '__fish_yadm_needs_command' -a 'upgrade'   -d 'Upgrade to version 2 of yadm directory structure'
+complete -x -c yadm -n '__fish_yadm_needs_command' -a 'version'   -d 'Print the version of yadm'
+
+complete -x -c yadm -n '__fish_yadm_needs_command' -a 'init' -d 'Initialize an empty repository'
+complete -x -c yadm -n '__fish_yadm_using_command init' -s f -d 'force to overwrite'
+complete -F -c yadm -n '__fish_yadm_using_command init' -s w -d 'set work-tree (default: $HOME)'
+
+complete -x -c yadm -n '__fish_yadm_needs_command' -a 'list' -d 'List tracked files at current directory'
+complete -x -c yadm -n '__fish_yadm_using_command list' -s a -d 'list all managed files instead'
+
+complete -x -c yadm -n '__fish_yadm_needs_command' -a 'encrypt' -d 'Encrypt files'
+complete -x -c yadm -n '__fish_yadm_needs_command' -a 'decrypt' -d 'Decrypt files'
+complete -x -c yadm -n '__fish_yadm_using_command decrypt' -s l -d 'list the files stored without extracting'
+
+complete -x -c yadm -n '__fish_yadm_needs_command' -a 'introspect' -d 'Report internal yadm data'
+complete -x -c yadm -n '__fish_yadm_using_command introspect' -a (printf -- '%s\n' 'commands configs repo switches') -d 'category'
+
+complete -x -c yadm -n '__fish_yadm_needs_command' -a 'gitconfig' -d 'Pass options to the git config command'
+complete -x -c yadm -n '__fish_yadm_needs_command' -a 'config'    -d 'Configure a setting'
+for name in (yadm introspect configs)
+    complete -x -c yadm -n '__fish_yadm_using_command config' -a '$name' -d 'yadm config'
+end
+
+# yadm universial options
+complete --force-files -c yadm -s Y -l yadm-dir       -d 'Override location of yadm directory'
+complete --force-files -c yadm      -l yadm-repo      -d 'Override location of yadm repository'
+complete --force-files -c yadm      -l yadm-config    -d 'Override location of yadm configuration file'
+complete --force-files -c yadm      -l yadm-encrypt   -d 'Override location of yadm encryption configuration'
+complete --force-files -c yadm      -l yadm-archive   -d 'Override location of yadm encrypted files archive'
+complete --force-files -c yadm      -l yadm-bootstrap -d 'Override location of yadm bootstrap program'
+
+# wraps git's autocomplete
+set -l GIT_DIR (yadm introspect repo)
+# setup the correct git-dir by appending it to git's argunment
+complete -c yadm -w "git --git-dir=$GIT_DIR"

completion/yadm.zsh_completion

@@ -1,46 +0,0 @@
-#compdef yadm
-_yadm(){
-    local -a _1st_arguments
-     _1st_arguments=(
-        'help:Display yadm command help'
-        'init:Initialize an empty repository'
-        'config:Configure a setting'
-        'list:List tracked files'
-        'alt:Create links for alternates'
-        'bootstrap:Execute $HOME/.config/yadm/bootstrap'
-        'encrypt:Encrypt files'
-        'decrypt:Decrypt files'
-        'perms:Fix perms for private files'
-        'add:git add'
-        'push:git push'
-        'pull:git pull'
-        'diff:git diff'
-        'checkout:git checkout'
-        'co:git co'
-        'commit:git commit'
-        'ci:git ci'
-        'status:git status'
-        'st:git st'
-        'reset:git reset'
-        'log:git log'
-    )
-
-    local context state line expl
-    local -A opt_args
-
-    _arguments '*:: :->subcmds' && return 0
-
-    if (( CURRENT == 1 )); then
-        _describe -t commands "yadm commands" _1st_arguments -V1
-        return
-    fi
-
-    case "$words[1]" in
-        *)
-            _arguments ':filenames:_files'
-        ;;
-    esac
-
-}
-
-_yadm "$@"

completion/zsh/_yadm

@@ -0,0 +1,180 @@
+#compdef yadm
+
+# This completion tries to fallback to git's completion for git commands.
+
+zstyle -T ':completion:*:yadm:argument-1:descriptions:' format && \
+    zstyle ':completion:*:yadm:argument-1:descriptions' format '%d:'
+zstyle -T ':completion:*:yadm:*:yadm' group-name && \
+    zstyle ':completion:*:yadm:*:yadm' group-name ''
+
+function _yadm-add(){
+  local -a yadm_options yadm_path
+  yadm_path="$(yadm rev-parse --show-toplevel)"
+  yadm_options=($(yadm status --porcelain=v1 |
+    awk -v yadm_path=${yadm_path} '{printf "%s/%s:%s\n",  yadm_path, $2, $1}' ))
+
+  _describe 'command' yadm_options
+  _files
+}
+
+function _yadm-checkout(){
+    _yadm-add
+}
+
+_yadm-alt() {
+    return 0
+}
+
+_yadm-bootstrap() {
+    return 0
+}
+
+_yadm-clone() {
+    _arguments \
+        '(--bootstrap --no-bootstrap)--bootstrap[force bootstrap, without prompt]' \
+        '(--bootstrap --no-bootstrap)--no-bootstrap[prevent bootstrap, without prompt]' \
+        '-f[force overwrite of existing repository]' \
+        '-w[yadm work tree path]: :_files -/'
+
+    local curcontext="${curcontext%:*:*}:git:"
+
+    words=("git" "${words[@]}") CURRENT=$((CURRENT + 1)) service=git _git
+}
+
+_yadm-config() {
+    # TODO: complete config names
+}
+
+_yadm-decrypt() {
+    _arguments \
+        '-l[list files]'
+}
+
+_yadm-encrypt() {
+    return 0
+}
+
+_yadm-enter() {
+    _arguments \
+        ':command: _command_names -e' \
+        '*::arguments: _normal'
+}
+
+_yadm-git-crypt() {
+    # TODO: complete git-crypt options
+}
+
+_yadm-help() {
+    return 0
+}
+
+_yadm-init() {
+    _arguments \
+        '-f[force overwrite of existing repository]' \
+        '-w[work tree path]: :_files -/'
+}
+
+_yadm-list() {
+    _arguments \
+        '-a[list all tracked files]'
+}
+
+_yadm-perms() {
+    return 0
+}
+
+_yadm-transcrypt() {
+    integer _ret=1
+    _call_function _ret _transcrypt
+    return _ret
+}
+
+_yadm-upgrade() {
+    _arguments \
+        '-f[force deinit of submodules]' \
+        ': '
+}
+
+_yadm-version() {
+    return 0
+}
+
+_yadm_commands() {
+    local -a commands=(
+        alt:'create links for alternates'
+        bootstrap:'execute bootstrap'
+        clone:'clone an existing yadm repository'
+        config:'configure an yadm setting'
+        decrypt:'decrypt files'
+        encrypt:'encrypt files'
+        enter:'run sub-shell with GIT variables set'
+        git-crypt:'run git-crypt commands for the yadm repository'
+        gitconfig:'run the git config command'
+        help:'display yadm help information'
+        init:'initialize an empty yadm repository'
+        list:'list files tracked by yadm'
+        perms:'fix perms for private files'
+        transcrypt:'run transcrypt commands for the yadm repository'
+        upgrade:'upgrade legacy yadm paths'
+        version:'show yadm version'
+    )
+
+    local oldcontext="$curcontext"
+    local curcontext="${curcontext%:*:*}:git:"
+
+    words=("git" "${words[-1]}") CURRENT=2 service=git _git
+
+    curcontext="$oldcontext"
+    _describe -t yadm "yadm commands" commands
+
+    return 0
+}
+
+_yadm() {
+    local curcontext=$curcontext state state_descr line
+    declare -A opt_args
+
+    _arguments -C \
+      '(-Y --yadm-dir)'{-Y,--yadm-dir}'[override the standard yadm directory]: :_files -/' \
+      '--yadm-data[override the standard yadm data directory]: :_files -/' \
+      '--yadm-repo[override the standard repo path]: :_files -/' \
+      '--yadm-config[override the standard config path]: :_files -/' \
+      '--yadm-encrypt[override the standard encrypt path]: :_files -/' \
+      '--yadm-archive[override the standard archive path]: :_files -/' \
+      '--yadm-bootstrap[override the standard bootstrap path]: :_files' \
+      '--help[display yadm help information]' \
+      '--version[show yadm version]' \
+      '(-): :->command' \
+      '(-)*:: :->option-or-argument' && return
+
+    local -a repo_args
+    (( $+opt_args[--yadm-repo] )) && repo_args+=(--yadm-repo "$opt_args[--yadm-repo]")
+    (( $+opt_args[--yadm-data] )) && repo_args+=(--yadm-data "$opt_args[--yadm-data]")
+    local -x GIT_DIR="$(_call_program gitdir yadm "${repo_args[@]}" introspect repo)"
+    [[ -z "$GIT_DIR" ]] && return 1
+
+    integer _ret=1
+    case $state in
+        (command)
+            _yadm_commands && _ret=0
+            ;;
+        (option-or-argument)
+            curcontext=${curcontext%:*:*}:yadm-${words[1]}:
+            if ! _call_function _ret _yadm-${words[1]}; then
+
+                # Translate gitconfig to use the regular completion for config
+                [[ ${words[1]} = "gitconfig" ]] && words[1]=config
+
+                words=("git" "${(@)words}")
+                CURRENT=$(( CURRENT + 1 ))
+
+                curcontext=${curcontext%:*:*}:git:
+                service=git _git && _ret=0
+            fi
+            ;;
+    esac
+
+    return _ret
+}
+
+(( $+functions[_git] )) && _yadm

contrib/bootstrap/bootstrap-in-dir

@@ -0,0 +1,24 @@
+#!/bin/bash
+
+# Save this file as ~/.config/yadm/bootstrap and make it executable. It will
+# execute all executable files (excluding templates and editor backups) in the
+# ~/.config/yadm/bootstrap.d directory when run.
+
+set -eu
+
+# Directory to look for bootstrap executables in
+BOOTSTRAP_D="${BASH_SOURCE[0]}.d"
+
+if [[ ! -d "$BOOTSTRAP_D" ]]; then
+    echo "Error: bootstrap directory '$BOOTSTRAP_D' not found" >&2
+    exit 1
+fi
+
+find -L "$BOOTSTRAP_D" -type f | sort | while IFS= read -r bootstrap; do
+    if [[ -x "$bootstrap" && ! "$bootstrap" =~ "##" && ! "$bootstrap" =~ "~$" ]]; then
+        if ! "$bootstrap"; then
+            echo "Error: bootstrap '$bootstrap' failed" >&2
+            exit 1
+        fi
+    fi
+done

contrib/commands/README.md

@@ -0,0 +1,9 @@
+## Contributed Commands
+
+Although these commands are available as part of the official
+**yadm** source tree, they have a somewhat different status. The intention is to
+keep interesting and potentially useful commands here, building a library of
+examples that might help others.
+
+I recommend *careful review* of any code from here before using it. No
+guarantees of code quality is assumed.

contrib/commands/yadm-untracked

@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+
+# To run: `yadm-untracked <config-file>`
+#
+# If you wish to create a YADM alias to run this as, for example `yadm untracked`
+# then the following command will add the alias:
+#     `yadm gitconfig alias.untracked '!<PATH>/yadm-untracked'`
+
+# Possible script improvements:
+# - Reduce the amount of configuration; I have not figured out a way to
+#   get rid of the non-recursive and ignore. The recursive list could be
+#   built from the directories that are present in `yadm list`
+
+# Configuration... The script looks at the following 3 arrays:
+#
+# yadm_tracked_recursively
+#     The directories and files in this list are searched recursively to build
+#     a list of files that you expect are tracked with `yadm`. Items in this
+#     list are relative to the root of your YADM repo (which is $HOME for most).
+
+# yadm_tracked_nonrecursively
+#     Same as above but don't search recursively
+#
+# ignore_files_and_dirs
+#     A list of directories and files that will not be reported as untracked if
+#     found in the above two searches.
+#
+# Example configuration file (uncomment it to use):
+# yadm_tracked_recursively=(
+#     bin .config .vim
+# )
+#
+# yadm_tracked_nonrecursively=(
+#     ~
+# )
+#
+# ignore_files_and_dirs=(
+#     .CFUserTextEncoding .DS_Store .config/gh
+#     .vim/autoload/plug.vim
+# )
+
+if [[ $# -ne 1 ]]; then
+    echo 'Usage: yadm-untracked <config-file>'
+    exit 1
+fi
+
+yadm_tracked_recursively=()
+yadm_tracked_nonrecursively=()
+ignore_files_and_dirs=()
+
+source $1
+
+root=`yadm enter echo '$GIT_WORK_TREE'`
+
+cd $root
+
+find_list=$(mktemp -t find_list)
+find ${yadm_tracked_recursively[*]} -type f >$find_list
+find ${yadm_tracked_nonrecursively[*]} -maxdepth 1 -type f |
+  awk "{sub(\"^\./\", \"\"); sub(\"^$root/\", \"\"); print }" >>$find_list
+sort -o $find_list $find_list
+
+yadm_list=$(mktemp -t yadm_list)
+yadm list >$yadm_list
+find ${ignore_files_and_dirs[*]} -type f >>$yadm_list
+sort -o $yadm_list $yadm_list
+
+# Show the files not in `yadm list`
+comm -23 $find_list $yadm_list
+
+rm -f $find_list $yadm_list

contrib/hooks/README.md

@@ -11,4 +11,4 @@ this is a place to share it.
 I recommend *careful review* of any code from here before using it. No
 guarantees of code quality is assumed.
 
-[hooks-help]: https://github.com/TheLocehiliosan/yadm/blob/master/yadm.md#hooks
+[hooks-help]: https://github.com/yadm-dev/yadm/blob/master/yadm.md#hooks

contrib/hooks/encrypt_with_checksums/README.md

@@ -0,0 +1,9 @@
+## Track checksums of encrypted files
+
+Contributed by Martin Zuther
+
+Hook         | Description
+----         | -----------
+post_encrypt | Collects the checksums of encrypted files, and stores them in .config/yadm/files.checksums
+post_list    | Prints the names of encrypted files
+post_status  | Reports untracked changes within encrypted files

contrib/hooks/encrypt_with_checksums/post_encrypt

@@ -0,0 +1,96 @@
+#!/usr/bin/env bash
+
+# yadm - Yet Another Dotfiles Manager
+# Copyright (C) 2015-2021 Tim Byrne and Martin Zuther
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+
+YADM_CHECKSUMS="$YADM_HOOK_DIR/files.checksums"
+WARNING_MESSAGE="No checksums were created"
+
+# unpack exported array; filenames including a newline character (\n)
+# are NOT supported
+OLD_IFS="$IFS"
+IFS=$'\n'
+YADM_ENCRYPT_INCLUDE_FILES=( $YADM_ENCRYPT_INCLUDE_FILES )
+IFS="$OLD_IFS"
+
+
+function get_checksum_command {
+    # check if "shasum" exists and supports the algorithm (which is
+    # tested by sending an empty string to "shasum")
+    if command -v "shasum" > /dev/null && printf "" | shasum --algorithm "256" &> /dev/null; then
+        printf "shasum --algorithm 256"
+    # check if "sha256sum" exists
+    elif command -v "sha256sum" > /dev/null; then
+        printf "sha256sum"
+    # check if "gsha256sum" exists
+    elif command -v "gsha256sum" > /dev/null; then
+        printf "gsha256sum"
+    else
+        # display warning in bright yellow
+        printf "\033[1;33m" >&2
+        printf "\nWARNING: \"shasum\", \"sha256sum\" and \"gsha256sum\" not found.   %s\n" "$WARNING_MESSAGE." >&2
+
+        # reset output color
+        printf "\033[0m" >&2
+
+        # signal error
+        return 1
+    fi
+}
+
+
+# get checksum command
+CHECKSUM_COMMAND=$(get_checksum_command)
+
+# no command found
+if (($?)); then
+    # return original exit status of yadm command
+    exit "$YADM_HOOK_EXIT"
+fi
+
+# empty (or create) checksum file
+true > "$YADM_CHECKSUMS"
+
+# calculate checksums for encrypted files
+for included in "${YADM_ENCRYPT_INCLUDE_FILES[@]}"; do
+    # highlight any errors in red
+    printf "\033[0;31m"
+
+    # calculate checksums
+    $CHECKSUM_COMMAND "$included" >> "$YADM_CHECKSUMS"
+    ERROR_CODE=$?
+
+    # reset output color
+    printf "\033[0m"
+
+    # handle errors
+    if (($ERROR_CODE)); then
+        # display warning in bright yellow
+        printf "\033[1;33m" >&2
+        printf "\nWARNING: an error occurred.  Please inspect the checksum file.\n" >&2
+
+        # reset output color
+        printf "\033[0m" >&2
+
+        # exit and signal error
+        exit $ERROR_CODE
+    fi
+done
+
+# announce success and return original exit status of yadm command
+printf "Wrote SHA-256 checksums: %s\n" "$YADM_CHECKSUMS"
+exit "$YADM_HOOK_EXIT"

contrib/hooks/encrypt_with_checksums/post_list

@@ -0,0 +1,69 @@
+#!/usr/bin/env bash
+
+# yadm - Yet Another Dotfiles Manager
+# Copyright (C) 2015-2021 Tim Byrne and Martin Zuther
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+YADM_CHECKSUMS="$YADM_HOOK_DIR/files.checksums"
+
+
+# is current directory on yadm's work path?
+# (adapted from https://unix.stackexchange.com/a/6438/122163)
+if [ "${PWD##$YADM_HOOK_WORK}" != "$PWD" ]; then
+    ON_WORK_PATH=1
+else
+    ON_WORK_PATH=0
+fi
+
+
+# list all files or only those in the subdirectories below?
+OPTION_LIST_ALL=0
+for argument in "${YADM_HOOK_FULL_COMMAND[@]}"; do
+    # mimick git ls-files by displaying all files when not on work
+    # path
+    if [ "$argument" = "-a" ] || [ $ON_WORK_PATH -eq 0 ]; then
+        OPTION_LIST_ALL=1
+        break
+    fi
+done
+
+
+# if there is no checksum file, exit with original status of yadm
+# command
+if [ ! -f "$YADM_CHECKSUMS" ]; then
+    exit "$YADM_HOOK_EXIT"
+fi
+
+# list encrypted files
+while IFS= read -r filename; do
+    # remove checksums from file names
+    filename="${filename##[a-zA-Z0-9]* }"
+
+    # list only files in the subdirectories below (i.e. files
+    # whose relative path doesn't begin with "../")
+    if [ $OPTION_LIST_ALL -eq 0 ]; then
+        REL_PATH=$(relative_path "$PWD" "$YADM_HOOK_WORK/$filename")
+
+        if [ "$REL_PATH" = "${REL_PATH##../}" ]; then
+            printf "%s\n" "$REL_PATH"
+        fi
+    # list all files
+    else
+        printf "%s\n" "$filename"
+    fi
+done < "$YADM_CHECKSUMS"
+
+# return original exit status of yadm command
+exit "$YADM_HOOK_EXIT"

contrib/hooks/encrypt_with_checksums/post_status

@@ -0,0 +1,100 @@
+#!/usr/bin/env bash
+
+# yadm - Yet Another Dotfiles Manager
+# Copyright (C) 2015-2021 Tim Byrne and Martin Zuther
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+
+YADM_CHECKSUMS="$YADM_HOOK_DIR/files.checksums"
+WARNING_MESSAGE="Checksums were not verified"
+
+# unpack exported array; filenames including a newline character (\n)
+# are NOT supported
+OLD_IFS="$IFS"
+IFS=$'\n'
+YADM_ENCRYPT_INCLUDE_FILES=( $YADM_ENCRYPT_INCLUDE_FILES )
+IFS="$OLD_IFS"
+
+
+function get_checksum_command {
+    # check if "shasum" exists and supports the algorithm (which is
+    # tested by sending an empty string to "shasum")
+    if command -v "shasum" > /dev/null && printf "" | shasum --algorithm "256" &> /dev/null; then
+        printf "shasum --algorithm 256"
+    # check if "sha256sum" exists
+    elif command -v "sha256sum" > /dev/null; then
+        printf "sha256sum"
+    # check if "gsha256sum" exists
+    elif command -v "gsha256sum" > /dev/null; then
+        printf "gsha256sum"
+    else
+        # display warning in bright yellow
+        printf "\033[1;33m" >&2
+        printf "\nWARNING: \"shasum\", \"sha256sum\" and \"gsha256sum\" not found.   %s\n" "$WARNING_MESSAGE." >&2
+
+        # reset output color
+        printf "\033[0m" >&2
+
+        # signal error
+        return 1
+    fi
+}
+
+
+# if there is no checksum file, exit with original status of yadm
+# command
+if [ ! -f "$YADM_CHECKSUMS" ]; then
+    exit "$YADM_HOOK_EXIT"
+fi
+
+# get checksum command
+CHECKSUM_COMMAND=$(get_checksum_command)
+
+# no command found
+if (($?)); then
+    # return original exit status of yadm command
+    exit "$YADM_HOOK_EXIT"
+fi
+
+# check encrypted files for differences and capture output and error
+# messages
+YADM_CHECKSUM_OUTPUT=$($CHECKSUM_COMMAND --check "$YADM_CHECKSUMS" 2>&1)
+ERROR_CODE=$?
+
+# handle mismatched checksums and errors
+if (($ERROR_CODE)); then
+    printf "\nSome SHA-256 sums do not match (or an error occurred):\n\n"
+
+    # display differing files and errors (highlighted in red)
+    printf "\033[0;31m"
+
+    while IFS= read -r line; do
+        # beautify output and get rid of unnecessary lines
+        line="${line%%*: [Oo][Kk]}"
+        line="${line%%: [Ff][Aa][Ii][Ll][Ee][Dd]}"
+        line="${line##*WARNING:*did NOT match}"
+
+        if [ -n "$line" ]; then
+            printf "%s\n" "$line"
+        fi
+    done <<< "$YADM_CHECKSUM_OUTPUT"
+
+    # reset output color
+    printf "\033[0m"
+
+    # display advice for differing files and signal error
+    printf "\nConsider running either \"yadm encrypt\" or \"yadm decrypt\".\n"
+    exit $ERROR_CODE
+fi

contrib/hooks/parsing_full_command_example/README.md

@@ -0,0 +1,7 @@
+## Example of parsing `$YADM_HOOK_FULL_COMMAND`
+
+Contributed by Tim Byrne
+
+Hook    | Description
+----    | -----------
+pre_log | Provides an example of parsing `$YADM_HOOK_FULL_COMMAND` in Bash

contrib/hooks/parsing_full_command_example/pre_log

@@ -0,0 +1,26 @@
+#!/bin/bash
+
+# yadm exposes all parameters of the command which triggers a hook. Those
+# parameters are exported as the environment variable YADM_HOOK_FULL_COMMAND.
+# Any spaces, tabs, or backslashes in those parameters are escaped with a
+# backslash. The function `parse_full_command()` is a demonstration of parsing
+# those values which may be escaped.
+
+function parse_full_command() {
+  local delim=$'\x1e' # ASCII Record Separator
+  local space=$'\x1f' # ASCII Unit Separator
+  local tab=$'\t'     # ASCII TAB
+  local cmd
+  cmd="$YADM_HOOK_FULL_COMMAND"
+  cmd="${cmd//\\ /$space}"      # swap escaped spaces for `1f`
+  cmd="${cmd//\\\\/\\}"         # fix escaped backslashes
+  cmd="${cmd//\\$tab/$tab}"     # fix escaped tabs
+  cmd="${cmd// /$delim}"        # convert space delimiters to `1c`
+  cmd="${cmd//$space/ }"        # convert `1f` back to spaces
+  # parse data into an array
+  IFS=$delim read -r -a full_cmd <<< "$cmd"
+}
+parse_full_command
+for param in "${full_cmd[@]}"; do
+  echo "Parameter: '$param'"
+done

docker-compose.yml

@@ -1,7 +0,0 @@
----
-version: '3'
-services:
-  testbed:
-    volumes:
-      - .:/yadm:ro
-    image: yadm/testbed:2019-09-25

pylintrc

@@ -1,17 +0,0 @@
-[BASIC]
-good-names=pytestmark
-
-[DESIGN]
-max-args=14
-max-locals=28
-max-attributes=8
-max-statements=65
-
-[SIMILARITIES]
-min-similarity-lines=6
-
-[MESSAGES CONTROL]
-disable=redefined-outer-name
-
-[TYPECHECK]
-ignored-modules=py

pyproject.toml

@@ -0,0 +1,31 @@
+[tool.pytest.ini_options]
+cache_dir = "/tmp"
+addopts = "-ra"
+markers = [
+  "deprecated", # marks tests for deprecated features (deselect with '-m "not deprecated"')
+]
+
+[tool.pylint.design]
+max-args = 14
+max-locals = 28
+max-attributes = 8
+max-statements = 65
+
+[tool.pylint.format]
+max-line-length = 120
+
+[tool.pylint."messages control"]
+disable = [
+  "redefined-outer-name",
+]
+
+[tool.pylint.similarities]
+ignore-imports = "yes"
+min-similarity-lines = 8
+
+[tool.black]
+line-length = 120
+
+[tool.isort]
+line_length = 120
+profile = "black"

pytest.ini

@@ -1,5 +0,0 @@
-[pytest]
-cache_dir = /tmp
-addopts = -ra
-markers =
-  deprecated: marks tests for deprecated features (deselect with '-m "not deprecated"')

test/Dockerfile

@@ -0,0 +1,73 @@
+FROM ubuntu:23.04
+MAINTAINER Tim Byrne <sultan@locehilios.com>
+
+# Shellcheck and esh versions
+ARG SC_VER=0.9.0
+ARG ESH_VER=0.3.2
+
+# Install prerequisites and configure UTF-8 locale
+RUN \
+  echo "en_US.UTF-8 UTF-8" > /etc/locale.gen \
+  && apt-get update \
+  && DEBIAN_FRONTEND=noninteractive \
+    apt-get install -y --no-install-recommends \
+    expect \
+    git \
+    gnupg \
+    locales \
+    lsb-release \
+    make \
+    man \
+    python3-pip \
+    vim-tiny \
+    xz-utils \
+  && rm -rf /var/lib/apt/lists/* \
+  && update-locale LANG='en_US.UTF-8' LANGUAGE='en_US:en' LC_ALL='en_US.UTF-8'
+
+ENV LANG='en_US.UTF-8' LANGUAGE='en_US:en' LC_ALL='en_US.UTF-8'
+
+# Convenience settings for the testbed's root account
+RUN echo 'set -o vi' >> /root/.bashrc
+
+# Create a flag to identify when running inside the yadm testbed
+RUN touch /.yadmtestbed
+
+# Install shellcheck
+ADD https://github.com/koalaman/shellcheck/releases/download/v$SC_VER/shellcheck-v$SC_VER.linux.x86_64.tar.xz /opt
+RUN cd /opt \
+  && tar xf shellcheck-v$SC_VER.linux.x86_64.tar.xz \
+  && rm -f shellcheck-v$SC_VER.linux.x86_64.tar.xz \
+  && ln -s /opt/shellcheck-v$SC_VER/shellcheck /usr/local/bin
+
+# Upgrade pip3 and install requirements
+COPY test/requirements.txt /tmp/requirements.txt
+RUN python3 -m pip install --break-system-packages --upgrade pip setuptools \
+  && python3 -m pip install --break-system-packages --upgrade -r /tmp/requirements.txt \
+  && rm -f /tmp/requirements
+
+# Install esh
+ADD https://raw.githubusercontent.com/jirutka/esh/v$ESH_VER/esh /usr/local/bin
+RUN chmod +x /usr/local/bin/esh
+
+# Create workdir and dummy Makefile to be used if no /yadm volume is mounted
+RUN mkdir /yadm \
+  && echo "test:" > /yadm/Makefile \
+  && echo "\t@echo 'The yadm project must be mounted at /yadm'" >> /yadm/Makefile \
+  && echo "\t@echo 'Try using a docker parameter like -v \"\$\$PWD:/yadm:ro\"'" >> /yadm/Makefile \
+  && echo "\t@false" >> /yadm/Makefile
+
+# Include released versions of yadm to test upgrades
+ADD https://raw.githubusercontent.com/yadm-dev/yadm/1.12.0/yadm /usr/local/bin/yadm-1.12.0
+ADD https://raw.githubusercontent.com/yadm-dev/yadm/2.5.0/yadm /usr/local/bin/yadm-2.5.0
+RUN chmod +x /usr/local/bin/yadm-*
+
+# Configure git to make it easier to test yadm manually
+RUN git config --system user.email "test@yadm.io" \
+  && git config --system user.name "Yadm Test"
+
+# /yadm will be the work directory for all tests
+# docker commands should mount the local yadm project as /yadm
+WORKDIR /yadm
+
+# By default, run all tests defined
+CMD make test

test/conftest.py

@@ -3,140 +3,188 @@
 import collections
 import contextlib
 import copy
-import distutils.dir_util  # pylint: disable=no-name-in-module,import-error
 import os
 import platform
 import pwd
-from subprocess import Popen, PIPE
+import shutil
+from subprocess import PIPE, Popen
+
+import py
 import pytest
 
 
-@pytest.fixture(scope='session')
+def pytest_addoption(parser):
+    """Add options to pytest"""
+    parser.addoption(
+        "--force-linters",
+        action="store_true",
+        default=False,
+        help="Run linters regardless of installed versions",
+    )
+
+
+@pytest.fixture(scope="session")
 def shellcheck_version():
     """Version of shellcheck supported"""
-    return '0.4.6'
+    return "0.9.0"
 
 
-@pytest.fixture(scope='session')
+@pytest.fixture(scope="session")
 def pylint_version():
     """Version of pylint supported"""
-    return '2.4.1'
+    return "2.17.0"
 
 
-@pytest.fixture(scope='session')
+@pytest.fixture(scope="session")
+def isort_version():
+    """Version of isort supported"""
+    return "5.12.0"
+
+
+@pytest.fixture(scope="session")
 def flake8_version():
     """Version of flake8 supported"""
-    return '3.7.8'
+    return "6.0.0"
 
 
-@pytest.fixture(scope='session')
+@pytest.fixture(scope="session")
+def black_version():
+    """Version of black supported"""
+    return "23.1.0"
+
+
+@pytest.fixture(scope="session")
 def yamllint_version():
     """Version of yamllint supported"""
-    return '1.17.0'
+    return "1.30.0"
 
 
-@pytest.fixture(scope='session')
+@pytest.fixture(scope="session")
 def tst_user():
     """Test session's user id"""
     return pwd.getpwuid(os.getuid()).pw_name
 
 
-@pytest.fixture(scope='session')
+@pytest.fixture(scope="session")
 def tst_host():
     """Test session's short hostname value"""
-    return platform.node().split('.')[0]
+    return platform.node().split(".")[0]
 
 
-@pytest.fixture(scope='session')
+@pytest.fixture(scope="session")
 def tst_distro(runner):
     """Test session's distro"""
-    distro = ''
+    distro = ""
     with contextlib.suppress(Exception):
-        run = runner(command=['lsb_release', '-si'], report=False)
+        run = runner(command=["lsb_release", "-si"], report=False)
         distro = run.out.strip()
     return distro
 
 
-@pytest.fixture(scope='session')
+@pytest.fixture(scope="session")
+def tst_distro_family(runner):
+    """Test session's distro_family"""
+    family = ""
+    with contextlib.suppress(Exception):
+        run = runner(command=["grep", "-oP", r"ID_LIKE=\K.+", "/etc/os-release"], report=False)
+        family = run.out.strip()
+    return family
+
+
+@pytest.fixture(scope="session")
 def tst_sys():
     """Test session's uname value"""
     return platform.system()
 
 
-@pytest.fixture(scope='session')
+@pytest.fixture(scope="session")
+def tst_arch():
+    """Test session's uname value"""
+    return platform.machine()
+
+
+@pytest.fixture(scope="session")
 def supported_commands():
     """List of supported commands
 
     This list should be updated every time yadm learns a new command.
     """
     return [
-        'alt',
-        'bootstrap',
-        'clean',
-        'clone',
-        'config',
-        'decrypt',
-        'encrypt',
-        'enter',
-        'gitconfig',
-        'help',
-        'init',
-        'introspect',
-        'list',
-        'perms',
-        'upgrade',
-        'version',
-        ]
+        "alt",
+        "bootstrap",
+        "clean",
+        "clone",
+        "config",
+        "decrypt",
+        "encrypt",
+        "enter",
+        "git-crypt",
+        "gitconfig",
+        "help",
+        "init",
+        "introspect",
+        "list",
+        "perms",
+        "transcrypt",
+        "upgrade",
+        "version",
+    ]
 
 
-@pytest.fixture(scope='session')
+@pytest.fixture(scope="session")
 def supported_configs():
     """List of supported config options
 
     This list should be updated every time yadm learns a new config.
     """
     return [
-        'local.class',
-        'local.hostname',
-        'local.os',
-        'local.user',
-        'yadm.alt-copy',
-        'yadm.auto-alt',
-        'yadm.auto-exclude',
-        'yadm.auto-perms',
-        'yadm.auto-private-dirs',
-        'yadm.git-program',
-        'yadm.gpg-perms',
-        'yadm.gpg-program',
-        'yadm.gpg-recipient',
-        'yadm.ssh-perms',
-        ]
+        "local.arch",
+        "local.class",
+        "local.hostname",
+        "local.os",
+        "local.user",
+        "yadm.alt-copy",
+        "yadm.auto-alt",
+        "yadm.auto-exclude",
+        "yadm.auto-perms",
+        "yadm.auto-private-dirs",
+        "yadm.cipher",
+        "yadm.git-program",
+        "yadm.gpg-perms",
+        "yadm.gpg-program",
+        "yadm.gpg-recipient",
+        "yadm.openssl-ciphername",
+        "yadm.openssl-old",
+        "yadm.openssl-program",
+        "yadm.ssh-perms",
+    ]
 
 
-@pytest.fixture(scope='session')
+@pytest.fixture(scope="session")
 def supported_switches():
     """List of supported switches
 
     This list should be updated every time yadm learns a new switch.
     """
     return [
-        '--yadm-archive',
-        '--yadm-bootstrap',
-        '--yadm-config',
-        '--yadm-dir',
-        '--yadm-encrypt',
-        '--yadm-repo',
-        '-Y',
-        ]
+        "--yadm-archive",
+        "--yadm-bootstrap",
+        "--yadm-config",
+        "--yadm-data",
+        "--yadm-dir",
+        "--yadm-encrypt",
+        "--yadm-repo",
+        "-Y",
+    ]
 
 
-@pytest.fixture(scope='session')
+@pytest.fixture(scope="session")
 def supported_local_configs(supported_configs):
     """List of supported local config options"""
-    return [c for c in supported_configs if c.startswith('local.')]
+    return [c for c in supported_configs if c.startswith("local.")]
 
 
-class Runner():
+class Runner:
     """Class for running commands
 
     Within yadm tests, this object should be used when running commands that
@@ -149,90 +197,76 @@ class Runner():
     Other instances of simply running commands should use os.system().
     """
 
-    def __init__(
-            self,
-            command,
-            inp=None,
-            shell=False,
-            cwd=None,
-            env=None,
-            expect=None,
-            report=True):
+    def __init__(self, command, inp=None, shell=False, cwd=None, env=None, expect=None, report=True):
         if shell:
-            self.command = ' '.join([str(cmd) for cmd in command])
+            self.command = " ".join([str(cmd) for cmd in command])
         else:
             self.command = command
+        if env is None:
+            env = {}
+        merged_env = os.environ.copy()
+        merged_env.update(env)
         self.inp = inp
         self.wrap(expect)
-        process = Popen(
+        with Popen(
             self.command,
             stdin=PIPE,
             stdout=PIPE,
             stderr=PIPE,
             shell=shell,
             cwd=cwd,
-            env=env,
-        )
-        input_bytes = self.inp
-        if self.inp:
-            input_bytes = self.inp.encode()
-        (out_bstream, err_bstream) = process.communicate(input=input_bytes)
-        self.out = out_bstream.decode()
-        self.err = err_bstream.decode()
-        self.code = process.wait()
+            env=merged_env,
+        ) as process:
+            input_bytes = self.inp
+            if self.inp:
+                input_bytes = self.inp.encode()
+            (out_bstream, err_bstream) = process.communicate(input=input_bytes)
+            self.out = out_bstream.decode()
+            self.err = err_bstream.decode()
+            self.code = process.wait()
         self.success = self.code == 0
         self.failure = self.code != 0
         if report:
             self.report()
 
     def __repr__(self):
-        return f'Runner({self.command})'
+        return f"Runner({self.command})"
 
     def report(self):
         """Print code/stdout/stderr"""
-        print(f'{self}')
-        print(f'  RUN: code:{self.code}')
+        print(f"{self}")
+        print(f"  RUN: code:{self.code}")
         if self.inp:
-            print(f'  RUN: input:\n{self.inp}')
-        print(f'  RUN: stdout:\n{self.out}')
-        print(f'  RUN: stderr:\n{self.err}')
+            print(f"  RUN: input:\n{self.inp}")
+        print(f"  RUN: stdout:\n{self.out}")
+        print(f"  RUN: stderr:\n{self.err}")
 
     def wrap(self, expect):
         """Wrap command with expect"""
         if not expect:
             return
-        cmdline = ' '.join([f'"{w}"' for w in self.command])
-        expect_script = f'set timeout 2\nspawn {cmdline}\n'
+        cmdline = " ".join([f'"{w}"' for w in self.command])
+        expect_script = f"set timeout 2\nspawn {cmdline}\n"
         for question, answer in expect:
-            expect_script += (
-                'expect {\n'
-                f'"{question}" {{send "{answer}\\r"}}\n'
-                'timeout {close;exit 128}\n'
-                '}\n')
-        expect_script += (
-            'expect eof\n'
-            'foreach {pid spawnid os_error_flag value} [wait] break\n'
-            'exit $value')
+            expect_script += "expect {\n" f'"{question}" {{send "{answer}\\r"}}\n' "timeout {close;exit 128}\n" "}\n"
+        expect_script += "expect eof\n" "foreach {pid spawnid os_error_flag value} [wait] break\n" "exit $value"
         self.inp = expect_script
-        print(f'EXPECT:{expect_script}')
-        self.command = ['expect']
+        print(f"EXPECT:{expect_script}")
+        self.command = ["expect"]
 
 
-@pytest.fixture(scope='session')
+@pytest.fixture(scope="session")
 def runner():
     """Class for running commands"""
     return Runner
 
 
-@pytest.fixture(scope='session')
+@pytest.fixture(scope="session")
 def config_git():
     """Configure global git configuration, if missing"""
-    os.system(
-        'git config user.name || '
-        'git config --global user.name "test"')
-    os.system(
-        'git config user.email || '
-        'git config --global user.email "test@test.test"')
+    os.system("git config init.defaultBranch || git config --global init.defaultBranch master")
+    os.system('git config user.name || git config --global user.name "test"')
+    os.system('git config user.email || git config --global user.email "test@test.test"')
 
 
 @pytest.fixture()
@@ -242,19 +276,19 @@ def repo_config(runner, paths):
     def query_func(key):
         """Query a yadm repo configuration value"""
         run = runner(
-            command=('git', 'config', '--local', key),
-            env={'GIT_DIR': paths.repo},
+            command=("git", "config", "--local", key),
+            env={"GIT_DIR": paths.repo},
             report=False,
-            )
+        )
         return run.out.rstrip()
 
     return query_func
 
 
-@pytest.fixture(scope='session')
+@pytest.fixture(scope="session")
 def yadm():
     """Path to yadm program to be tested"""
-    full_path = os.path.realpath('yadm')
+    full_path = os.path.realpath("yadm")
     assert os.path.isfile(full_path), "yadm program file isn't present"
     return full_path
 
@@ -262,55 +296,75 @@ def yadm():
 @pytest.fixture()
 def paths(tmpdir, yadm):
     """Function scoped test paths"""
-    dir_root = tmpdir.mkdir('root')
-    dir_work = dir_root.mkdir('work')
-    dir_yadm = dir_root.mkdir('yadm')
-    dir_repo = dir_yadm.mkdir('repo.git')
-    dir_hooks = dir_yadm.mkdir('hooks')
-    dir_remote = dir_root.mkdir('remote')
-    file_archive = dir_yadm.join('files.gpg')
-    file_bootstrap = dir_yadm.join('bootstrap')
-    file_config = dir_yadm.join('config')
-    file_encrypt = dir_yadm.join('encrypt')
+
+    dir_root = tmpdir.mkdir("root")
+    dir_remote = dir_root.mkdir("remote")
+    dir_work = dir_root.mkdir("work")
+    dir_xdg_data = dir_root.mkdir("xdg_data")
+    dir_xdg_home = dir_root.mkdir("xdg_home")
+    dir_data = dir_xdg_data.mkdir("yadm")
+    dir_yadm = dir_xdg_home.mkdir("yadm")
+    dir_hooks = dir_yadm.mkdir("hooks")
+    dir_repo = dir_data.mkdir("repo.git")
+    file_archive = dir_data.join("archive")
+    file_bootstrap = dir_yadm.join("bootstrap")
+    file_config = dir_yadm.join("config")
+    file_encrypt = dir_yadm.join("encrypt")
     paths = collections.namedtuple(
-        'Paths', [
-            'pgm',
-            'root',
-            'work',
-            'yadm',
-            'repo',
-            'hooks',
-            'remote',
-            'archive',
-            'bootstrap',
-            'config',
-            'encrypt',
-            ])
+        "Paths",
+        [
+            "pgm",
+            "root",
+            "remote",
+            "work",
+            "xdg_data",
+            "xdg_home",
+            "data",
+            "yadm",
+            "hooks",
+            "repo",
+            "archive",
+            "bootstrap",
+            "config",
+            "encrypt",
+        ],
+    )
+    os.environ["XDG_CONFIG_HOME"] = str(dir_xdg_home)
+    os.environ["XDG_DATA_HOME"] = str(dir_xdg_data)
     return paths(
         yadm,
         dir_root,
-        dir_work,
-        dir_yadm,
-        dir_repo,
-        dir_hooks,
         dir_remote,
+        dir_work,
+        dir_xdg_data,
+        dir_xdg_home,
+        dir_data,
+        dir_yadm,
+        dir_hooks,
+        dir_repo,
         file_archive,
         file_bootstrap,
         file_config,
         file_encrypt,
-        )
+    )
 
 
 @pytest.fixture()
-def yadm_y(paths):
+def yadm_cmd(paths):
     """Generate custom command_list function"""
+
     def command_list(*args):
         """Produce params for running yadm with -Y"""
-        return [paths.pgm, '-Y', str(paths.yadm)] + list(args)
+        return [paths.pgm] + list(args)
+
     return command_list
 
 
-class DataFile():
+class NoRelativePath(Exception):
+    """Exception when finding relative paths"""
+
+
+class DataFile:
     """Datafile object"""
 
     def __init__(self, path, tracked=True, private=False):
@@ -329,7 +383,7 @@ class DataFile():
         """Relative path property"""
         if self.__parent:
             return self.__parent.join(self.path)
-        raise BaseException('Unable to provide relative path, no parent')
+        raise NoRelativePath("Unable to provide relative path, no parent")
 
     @property
     def tracked(self):
@@ -346,22 +400,18 @@ class DataFile():
         self.__parent = parent
 
 
-class DataSet():
+class DataSet:
     """Dataset object"""
 
     def __init__(self):
-        self.__files = list()
-        self.__dirs = list()
-        self.__tracked_dirs = list()
-        self.__private_dirs = list()
+        self.__files = []
+        self.__dirs = []
+        self.__tracked_dirs = []
+        self.__private_dirs = []
         self.__relpath = None
 
     def __repr__(self):
-        return (
-            f'[DS with {len(self)} files; '
-            f'{len(self.tracked)} tracked, '
-            f'{len(self.private)} private]'
-            )
+        return f"[DS with {len(self)} files; " f"{len(self.tracked)} tracked, " f"{len(self.private)} private]"
 
     def __iter__(self):
         return iter(self.__files)
@@ -399,17 +449,17 @@ class DataSet():
     @property
     def plain_dirs(self):
         """List of directories in DataSet not starting with '.'"""
-        return [d for d in self.dirs if not d.startswith('.')]
+        return [d for d in self.dirs if not d.startswith(".")]
 
     @property
     def hidden_dirs(self):
         """List of directories in DataSet starting with '.'"""
-        return [d for d in self.dirs if d.startswith('.')]
+        return [d for d in self.dirs if d.startswith(".")]
 
     @property
     def tracked_dirs(self):
         """List of directories in DataSet not starting with '.'"""
-        return [d for d in self.__tracked_dirs if not d.startswith('.')]
+        return [d for d in self.__tracked_dirs if not d.startswith(".")]
 
     @property
     def private_dirs(self):
@@ -439,32 +489,23 @@ class DataSet():
             datafile.relative_to(self.__relpath)
 
 
-@pytest.fixture(scope='session')
+@pytest.fixture(scope="session")
 def ds1_dset(tst_sys):
     """Meta-data for dataset one files"""
     dset = DataSet()
-    dset.add_file('t1')
-    dset.add_file('d1/t2')
-    dset.add_file(f'test_alt##S')
-    dset.add_file(f'test_alt##S.H')
-    dset.add_file(f'test_alt##S.H.U')
-    dset.add_file(f'test_alt##C.S.H.U')
-    dset.add_file(f'test alt/test alt##S')
-    dset.add_file(f'test alt/test alt##S.H')
-    dset.add_file(f'test alt/test alt##S.H.U')
-    dset.add_file(f'test alt/test alt##C.S.H.U')
-    dset.add_file(f'test_alt_copy##{tst_sys}')
-    dset.add_file(f'test_alt_copy##os.{tst_sys}')
-    dset.add_file('u1', tracked=False)
-    dset.add_file('d2/u2', tracked=False)
-    dset.add_file('.ssh/p1', tracked=False, private=True)
-    dset.add_file('.ssh/.p2', tracked=False, private=True)
-    dset.add_file('.gnupg/p3', tracked=False, private=True)
-    dset.add_file('.gnupg/.p4', tracked=False, private=True)
+    dset.add_file("t1")
+    dset.add_file("d1/t2")
+    dset.add_file(f"test_alt_copy##os.{tst_sys}")
+    dset.add_file("u1", tracked=False)
+    dset.add_file("d2/u2", tracked=False)
+    dset.add_file(".ssh/p1", tracked=False, private=True)
+    dset.add_file(".ssh/.p2", tracked=False, private=True)
+    dset.add_file(".gnupg/p3", tracked=False, private=True)
+    dset.add_file(".gnupg/.p4", tracked=False, private=True)
     return dset
 
 
-@pytest.fixture(scope='session')
+@pytest.fixture(scope="session")
 def ds1_data(tmpdir_factory, config_git, ds1_dset, runner):
     """A set of test data, worktree & repo"""
     # pylint: disable=unused-argument
@@ -472,65 +513,40 @@ def ds1_data(tmpdir_factory, config_git, ds1_dset, runner):
     # @pytest.mark.usefixtures('config_git')
     # cannot be applied to another fixture.
 
-    data = tmpdir_factory.mktemp('ds1')
+    data = tmpdir_factory.mktemp("ds1")
 
-    work = data.mkdir('work')
+    work = data.mkdir("work")
     for datafile in ds1_dset:
         work.join(datafile.path).write(datafile.path, ensure=True)
 
-    repo = data.mkdir('repo.git')
+    repo = data.mkdir("repo.git")
     env = os.environ.copy()
-    env['GIT_DIR'] = str(repo)
-    runner(
-        command=['git', 'init', '--shared=0600', '--bare', str(repo)],
-        report=False)
-    runner(
-        command=['git', 'config', 'core.bare', 'false'],
-        env=env,
-        report=False)
-    runner(
-        command=['git', 'config', 'status.showUntrackedFiles', 'no'],
-        env=env,
-        report=False)
-    runner(
-        command=['git', 'config', 'yadm.managed', 'true'],
-        env=env,
-        report=False)
-    runner(
-        command=['git', 'config', 'core.worktree', str(work)],
-        env=env,
-        report=False)
-    runner(
-        command=['git', 'add'] +
-        [str(work.join(f.path)) for f in ds1_dset if f.tracked],
-        env=env)
-    runner(
-        command=['git', 'commit', '--allow-empty', '-m', 'Initial commit'],
-        env=env,
-        report=False)
+    env["GIT_DIR"] = str(repo)
+    runner(command=["git", "init", "--shared=0600", "--bare", str(repo)], report=False)
+    runner(command=["git", "config", "core.bare", "false"], env=env, report=False)
+    runner(command=["git", "config", "status.showUntrackedFiles", "no"], env=env, report=False)
+    runner(command=["git", "config", "yadm.managed", "true"], env=env, report=False)
+    runner(command=["git", "config", "core.worktree", str(work)], env=env, report=False)
+    runner(command=["git", "add"] + [str(work.join(f.path)) for f in ds1_dset if f.tracked], env=env)
+    runner(command=["git", "commit", "--allow-empty", "-m", "Initial commit"], env=env, report=False)
 
-    data = collections.namedtuple('Data', ['work', 'repo'])
+    data = collections.namedtuple("Data", ["work", "repo"])
     return data(work, repo)
 
 
 @pytest.fixture()
 def ds1_work_copy(ds1_data, paths):
     """Function scoped copy of ds1_data.work"""
-    distutils.dir_util.copy_tree(  # pylint: disable=no-member
-        str(ds1_data.work), str(paths.work))
+    shutil.copytree(str(ds1_data.work), str(paths.work), dirs_exist_ok=True)
 
 
 @pytest.fixture()
 def ds1_repo_copy(runner, ds1_data, paths):
     """Function scoped copy of ds1_data.repo"""
-    distutils.dir_util.copy_tree(  # pylint: disable=no-member
-        str(ds1_data.repo), str(paths.repo))
+    shutil.copytree(str(ds1_data.repo), str(paths.repo), dirs_exist_ok=True)
     env = os.environ.copy()
-    env['GIT_DIR'] = str(paths.repo)
-    runner(
-        command=['git', 'config', 'core.worktree', str(paths.work)],
-        env=env,
-        report=False)
+    env["GIT_DIR"] = str(paths.repo)
+    runner(command=["git", "config", "core.worktree", str(paths.work)], env=env, report=False)
 
 
 @pytest.fixture()
@@ -553,3 +569,29 @@ def ds1(ds1_work_copy, paths, ds1_dset):
     dscopy = copy.deepcopy(ds1_dset)
     dscopy.relative_to(copy.deepcopy(paths.work))
     return dscopy
+
+
+@pytest.fixture(scope="session")
+def gnupg(tmpdir_factory, runner):
+    """Location of GNUPGHOME"""
+
+    def register_gpg_password(password):
+        """Publish a new GPG mock password"""
+        py.path.local("/tmp/mock-password").write(password)
+
+    home = tmpdir_factory.mktemp("gnupghome")
+    home.chmod(0o700)
+    conf = home.join("gpg.conf")
+    conf.write("no-secmem-warning\n")
+    conf.chmod(0o600)
+    agentconf = home.join("gpg-agent.conf")
+    agentconf.write(f'pinentry-program {os.path.abspath("test/pinentry-mock")}\n' "max-cache-ttl 0\n")
+    agentconf.chmod(0o600)
+    data = collections.namedtuple("GNUPG", ["home", "pw"])
+    env = os.environ.copy()
+    env["GNUPGHOME"] = home
+
+    # this pre-populates std files in the GNUPGHOME
+    runner(["gpg", "-k"], env=env)
+
+    return data(home, register_gpg_password)

test/pinentry-mock

@@ -0,0 +1,20 @@
+#!/bin/bash
+# This program is a custom mock pinentry program
+# It uses whatever password is found in the /tmp directory
+# If the password is empty, replies CANCEL causing an error to similate invalid
+# credentials
+echo "OK Pleased to meet you"
+while read -r line; do
+  if [[ $line =~ GETPIN ]]; then
+    password="$(cat /tmp/mock-password 2>/dev/null)"
+    if [ -n "$password" ]; then
+      echo -n "D "
+      echo "$password"
+      echo "OK";
+    else
+      echo "CANCEL";
+    fi
+  else
+    echo "OK";
+  fi
+done

test/pylintrc

@@ -1 +0,0 @@
-../pylintrc

test/requirements.txt

@@ -0,0 +1,8 @@
+black==23.1.0
+envtpl
+flake8==6.0.0
+isort==5.12.0
+j2cli
+pylint==2.17.0
+pytest==7.2.2
+yamllint==1.30.0

test/test_alt.py

@@ -1,6 +1,7 @@
 """Test alt"""
 import os
 import string
+
 import py
 import pytest
 import utils
@@ -8,113 +9,156 @@ import utils
 TEST_PATHS = [utils.ALT_FILE1, utils.ALT_FILE2, utils.ALT_DIR]
 
 
-@pytest.mark.usefixtures('ds1_copy')
-@pytest.mark.parametrize('yadm_alt', [True, False], ids=['alt', 'worktree'])
+@pytest.mark.usefixtures("ds1_copy")
+@pytest.mark.parametrize("yadm_alt", [True, False], ids=["alt", "worktree"])
 @pytest.mark.parametrize(
-    'tracked,encrypt,exclude', [
+    "tracked,encrypt,exclude",
+    [
         (False, False, False),
         (True, False, False),
         (False, True, False),
         (False, True, True),
-    ], ids=['untracked', 'tracked', 'encrypted', 'excluded'])
-def test_alt_source(
-        runner, paths,
-        tracked, encrypt, exclude,
-        yadm_alt):
+    ],
+    ids=["untracked", "tracked", "encrypted", "excluded"],
+)
+def test_alt_source(runner, paths, tracked, encrypt, exclude, yadm_alt):
     """Test yadm alt operates on all expected sources of alternates"""
-    yadm_dir = setup_standard_yadm_dir(paths)
+    yadm_dir, yadm_data = setup_standard_yadm_dir(paths)
 
     utils.create_alt_files(
-        paths, '##default', tracked=tracked, encrypt=encrypt, exclude=exclude,
-        yadm_alt=yadm_alt, yadm_dir=yadm_dir)
-    run = runner([paths.pgm, '-Y', yadm_dir, 'alt'])
+        paths, "##default", tracked=tracked, encrypt=encrypt, exclude=exclude, yadm_alt=yadm_alt, yadm_dir=yadm_dir
+    )
+    run = runner([paths.pgm, "-Y", yadm_dir, "--yadm-data", yadm_data, "alt"])
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
     linked = utils.parse_alt_output(run.out)
 
-    basepath = yadm_dir.join('alt') if yadm_alt else paths.work
+    basepath = yadm_dir.join("alt") if yadm_alt else paths.work
 
     for link_path in TEST_PATHS:
-        source_file_content = link_path + '##default'
+        source_file_content = link_path + "##default"
         source_file = basepath.join(source_file_content)
         link_file = paths.work.join(link_path)
         if tracked or (encrypt and not exclude):
             assert link_file.islink()
-            target = py.path.local(link_file.readlink())
+            target = py.path.local(os.path.realpath(link_file))
             if target.isfile():
                 assert link_file.read() == source_file_content
                 assert str(source_file) in linked
             else:
-                assert link_file.join(
-                    utils.CONTAINED).read() == source_file_content
+                assert link_file.join(utils.CONTAINED).read() == source_file_content
                 assert str(source_file) in linked
         else:
             assert not link_file.exists()
             assert str(source_file) not in linked
 
 
-@pytest.mark.usefixtures('ds1_copy')
-@pytest.mark.parametrize('suffix', [
-    '##default',
-    '##o.$tst_sys', '##os.$tst_sys',
-    '##d.$tst_distro', '##distro.$tst_distro',
-    '##c.$tst_class', '##class.$tst_class',
-    '##h.$tst_host', '##hostname.$tst_host',
-    '##u.$tst_user', '##user.$tst_user',
-    ])
-def test_alt_conditions(
-        runner, paths,
-        tst_sys, tst_distro, tst_host, tst_user, suffix):
+@pytest.mark.usefixtures("ds1_copy")
+@pytest.mark.parametrize("yadm_alt", [True, False], ids=["alt", "worktree"])
+def test_relative_link(runner, paths, yadm_alt):
+    """Confirm links created are relative"""
+    yadm_dir, yadm_data = setup_standard_yadm_dir(paths)
+
+    utils.create_alt_files(
+        paths, "##default", tracked=True, encrypt=False, exclude=False, yadm_alt=yadm_alt, yadm_dir=yadm_dir
+    )
+    run = runner([paths.pgm, "-Y", yadm_dir, "--yadm-data", yadm_data, "alt"])
+    assert run.success
+    assert run.err == ""
+
+    basepath = yadm_dir.join("alt") if yadm_alt else paths.work
+
+    for link_path in TEST_PATHS:
+        source_file_content = link_path + "##default"
+        source_file = basepath.join(source_file_content)
+        link_file = paths.work.join(link_path)
+        link = link_file.readlink()
+        relpath = os.path.relpath(source_file, start=os.path.dirname(link_file))
+        assert link == relpath
+
+
+@pytest.mark.usefixtures("ds1_copy")
+@pytest.mark.parametrize(
+    "suffix",
+    [
+        "##default",
+        "##default,e.txt",
+        "##default,extension.txt",
+        "##a.$tst_arch",
+        "##arch.$tst_arch",
+        "##o.$tst_sys",
+        "##os.$tst_sys",
+        "##d.$tst_distro",
+        "##distro.$tst_distro",
+        "##f.$tst_distro_family",
+        "##distro_family.$tst_distro_family",
+        "##c.$tst_class",
+        "##class.$tst_class",
+        "##h.$tst_host",
+        "##hostname.$tst_host",
+        "##u.$tst_user",
+        "##user.$tst_user",
+    ],
+)
+def test_alt_conditions(runner, paths, tst_arch, tst_sys, tst_distro, tst_distro_family, tst_host, tst_user, suffix):
     """Test conditions supported by yadm alt"""
-    yadm_dir = setup_standard_yadm_dir(paths)
+    yadm_dir, yadm_data = setup_standard_yadm_dir(paths)
 
     # set the class
-    tst_class = 'testclass'
-    utils.set_local(paths, 'class', tst_class)
+    tst_class = "testclass"
+    utils.set_local(paths, "class", tst_class + ".before")
+    utils.set_local(paths, "class", tst_class, add=True)
+    utils.set_local(paths, "class", tst_class + ".after", add=True)
 
     suffix = string.Template(suffix).substitute(
+        tst_arch=tst_arch,
         tst_sys=tst_sys,
         tst_distro=tst_distro,
+        tst_distro_family=tst_distro_family,
         tst_class=tst_class,
         tst_host=tst_host,
         tst_user=tst_user,
     )
 
     utils.create_alt_files(paths, suffix)
-    run = runner([paths.pgm, '-Y', yadm_dir, 'alt'])
+    run = runner([paths.pgm, "-Y", yadm_dir, "--yadm-data", yadm_data, "alt"])
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
     linked = utils.parse_alt_output(run.out)
 
     for link_path in TEST_PATHS:
         source_file = link_path + suffix
         assert paths.work.join(link_path).islink()
-        target = py.path.local(paths.work.join(link_path).readlink())
+        target = py.path.local(os.path.realpath(paths.work.join(link_path)))
         if target.isfile():
             assert paths.work.join(link_path).read() == source_file
             assert str(paths.work.join(source_file)) in linked
         else:
-            assert paths.work.join(link_path).join(
-                utils.CONTAINED).read() == source_file
+            assert paths.work.join(link_path).join(utils.CONTAINED).read() == source_file
             assert str(paths.work.join(source_file)) in linked
 
 
-@pytest.mark.usefixtures('ds1_copy')
+@pytest.mark.usefixtures("ds1_copy")
+@pytest.mark.parametrize("kind", ["default", "", None, "envtpl", "j2cli", "j2", "esh"])
 @pytest.mark.parametrize(
-    'kind', ['default', '', None, 'envtpl', 'j2cli', 'j2'])
-@pytest.mark.parametrize('label', ['t', 'template', 'yadm', ])
-def test_alt_templates(
-        runner, paths, kind, label):
+    "label",
+    [
+        "t",
+        "template",
+        "yadm",
+    ],
+)
+def test_alt_templates(runner, paths, kind, label):
     """Test templates supported by yadm alt"""
-    yadm_dir = setup_standard_yadm_dir(paths)
+    yadm_dir, yadm_data = setup_standard_yadm_dir(paths)
 
-    suffix = f'##{label}.{kind}'
+    suffix = f"##{label}.{kind}"
     if kind is None:
-        suffix = f'##{label}'
+        suffix = f"##{label}"
     utils.create_alt_files(paths, suffix)
-    run = runner([paths.pgm, '-Y', yadm_dir, 'alt'])
+    run = runner([paths.pgm, "-Y", yadm_dir, "--yadm-data", yadm_data, "alt"])
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
     created = utils.parse_alt_output(run.out, linked=False)
 
     for created_path in TEST_PATHS:
@@ -125,41 +169,40 @@ def test_alt_templates(
             assert str(paths.work.join(source_file)) in created
 
 
-@pytest.mark.usefixtures('ds1_copy')
-@pytest.mark.parametrize('autoalt', [None, 'true', 'false'])
-def test_auto_alt(runner, yadm_y, paths, autoalt):
+@pytest.mark.usefixtures("ds1_copy")
+@pytest.mark.parametrize("autoalt", [None, "true", "false"])
+def test_auto_alt(runner, yadm_cmd, paths, autoalt):
     """Test auto alt"""
 
     # set the value of auto-alt
     if autoalt:
-        os.system(' '.join(yadm_y('config', 'yadm.auto-alt', autoalt)))
+        os.system(" ".join(yadm_cmd("config", "yadm.auto-alt", autoalt)))
 
-    utils.create_alt_files(paths, '##default')
-    run = runner(yadm_y('status'))
+    utils.create_alt_files(paths, "##default")
+    run = runner(yadm_cmd("status"))
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
     linked = utils.parse_alt_output(run.out)
 
     for link_path in TEST_PATHS:
-        source_file = link_path + '##default'
-        if autoalt == 'false':
+        source_file = link_path + "##default"
+        if autoalt == "false":
             assert not paths.work.join(link_path).exists()
         else:
             assert paths.work.join(link_path).islink()
-            target = py.path.local(paths.work.join(link_path).readlink())
+            target = py.path.local(os.path.realpath(paths.work.join(link_path)))
             if target.isfile():
                 assert paths.work.join(link_path).read() == source_file
                 # no linking output when run via auto-alt
                 assert str(paths.work.join(source_file)) not in linked
             else:
-                assert paths.work.join(link_path).join(
-                    utils.CONTAINED).read() == source_file
+                assert paths.work.join(link_path).join(utils.CONTAINED).read() == source_file
                 # no linking output when run via auto-alt
                 assert str(paths.work.join(source_file)) not in linked
 
 
-@pytest.mark.usefixtures('ds1_copy')
-def test_stale_link_removal(runner, yadm_y, paths):
+@pytest.mark.usefixtures("ds1_copy")
+def test_stale_link_removal(runner, yadm_cmd, paths):
     """Stale links to alternative files are removed
 
     This test ensures that when an already linked alternative becomes invalid
@@ -167,49 +210,48 @@ def test_stale_link_removal(runner, yadm_y, paths):
     """
 
     # set the class
-    tst_class = 'testclass'
-    utils.set_local(paths, 'class', tst_class)
+    tst_class = "testclass"
+    utils.set_local(paths, "class", tst_class)
 
     # create files which match the test class
-    utils.create_alt_files(paths, f'##class.{tst_class}')
+    utils.create_alt_files(paths, f"##class.{tst_class}")
 
     # run alt to trigger linking
-    run = runner(yadm_y('alt'))
+    run = runner(yadm_cmd("alt"))
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
     linked = utils.parse_alt_output(run.out)
 
     # assert the proper linking has occurred
     for stale_path in TEST_PATHS:
-        source_file = stale_path + '##class.' + tst_class
+        source_file = stale_path + "##class." + tst_class
         assert paths.work.join(stale_path).islink()
-        target = py.path.local(paths.work.join(stale_path).readlink())
+        target = py.path.local(os.path.realpath(paths.work.join(stale_path)))
         if target.isfile():
             assert paths.work.join(stale_path).read() == source_file
             assert str(paths.work.join(source_file)) in linked
         else:
-            assert paths.work.join(stale_path).join(
-                utils.CONTAINED).read() == source_file
+            assert paths.work.join(stale_path).join(utils.CONTAINED).read() == source_file
             assert str(paths.work.join(source_file)) in linked
 
     # change the class so there are no valid alternates
-    utils.set_local(paths, 'class', 'changedclass')
+    utils.set_local(paths, "class", "changedclass")
 
     # run alt to trigger linking
-    run = runner(yadm_y('alt'))
+    run = runner(yadm_cmd("alt"))
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
     linked = utils.parse_alt_output(run.out)
 
     # assert the linking is removed
     for stale_path in TEST_PATHS:
-        source_file = stale_path + '##class.' + tst_class
+        source_file = stale_path + "##class." + tst_class
         assert not paths.work.join(stale_path).exists()
         assert str(paths.work.join(source_file)) not in linked
 
 
-@pytest.mark.usefixtures('ds1_repo_copy')
-def test_template_overwrite_symlink(runner, yadm_y, paths, tst_sys):
+@pytest.mark.usefixtures("ds1_repo_copy")
+def test_template_overwrite_symlink(runner, yadm_cmd, paths, tst_sys):
     """Remove symlinks before processing a template
 
     If a symlink is in the way of the output of a template, the target of the
@@ -217,43 +259,44 @@ def test_template_overwrite_symlink(runner, yadm_y, paths, tst_sys):
     be removed just before processing a template.
     """
 
-    target = paths.work.join(f'test_link##os.{tst_sys}')
-    target.write('target')
+    target = paths.work.join(f"test_link##os.{tst_sys}")
+    target.write("target")
 
-    link = paths.work.join('test_link')
+    link = paths.work.join("test_link")
     link.mksymlinkto(target, absolute=1)
 
-    template = paths.work.join('test_link##template.default')
-    template.write('test-data')
+    template = paths.work.join("test_link##template.default")
+    template.write("test-data")
 
-    run = runner(yadm_y('add', target, template))
+    run = runner(yadm_cmd("add", target, template))
     assert run.success
-    assert run.err == ''
-    assert run.out == ''
+    assert run.err == ""
+    assert run.out == ""
     assert not link.islink()
-    assert target.read().strip() == 'target'
-    assert link.read().strip() == 'test-data'
+    assert target.read().strip() == "target"
+    assert link.read().strip() == "test-data"
 
 
-@pytest.mark.usefixtures('ds1_copy')
-@pytest.mark.parametrize('style', ['symlink', 'template'])
+@pytest.mark.usefixtures("ds1_copy")
+@pytest.mark.parametrize("style", ["symlink", "template"])
 def test_ensure_alt_path(runner, paths, style):
     """Test that directories are created before making alternates"""
-    yadm_dir = setup_standard_yadm_dir(paths)
-    suffix = 'default' if style == 'symlink' else 'template'
-    filename = 'a/b/c/file'
-    source = yadm_dir.join(f'alt/{filename}##{suffix}')
-    source.write('test-data', ensure=True)
-    run = runner([paths.pgm, '-Y', yadm_dir, 'add', source])
+    yadm_dir, yadm_data = setup_standard_yadm_dir(paths)
+    suffix = "default" if style == "symlink" else "template"
+    filename = "a/b/c/file"
+    source = yadm_dir.join(f"alt/{filename}##{suffix}")
+    source.write("test-data", ensure=True)
+    run = runner([paths.pgm, "-Y", yadm_dir, "--yadm-data", yadm_data, "add", source])
     assert run.success
-    assert run.err == ''
-    assert run.out == ''
-    assert paths.work.join(filename).read().strip() == 'test-data'
+    assert run.err == ""
+    assert run.out == ""
+    assert paths.work.join(filename).read().strip() == "test-data"
 
 
 def setup_standard_yadm_dir(paths):
     """Configure a yadm home within the work tree"""
-    std_yadm_dir = paths.work.mkdir('.config').mkdir('yadm')
-    std_yadm_dir.join('repo.git').mksymlinkto(paths.repo, absolute=1)
-    std_yadm_dir.join('encrypt').mksymlinkto(paths.encrypt, absolute=1)
-    return std_yadm_dir
+    std_yadm_dir = paths.work.mkdir(".config").mkdir("yadm")
+    std_yadm_data = paths.work.mkdir(".local").mkdir("share").mkdir("yadm")
+    std_yadm_data.join("repo.git").mksymlinkto(paths.repo, absolute=1)
+    std_yadm_dir.join("encrypt").mksymlinkto(paths.encrypt, absolute=1)
+    return std_yadm_dir, std_yadm_data

test/test_alt_copy.py

@@ -1,61 +1,46 @@
 """Test yadm.alt-copy"""
 
 import os
+
 import pytest
 
 
 @pytest.mark.parametrize(
-    'cygwin',
-    [pytest.param(True, marks=pytest.mark.deprecated), False],
-    ids=['cygwin', 'no-cygwin'])
-@pytest.mark.parametrize(
-    'compatibility', [True, False], ids=['compat', 'no-compat'])
-@pytest.mark.parametrize(
-    'setting, expect_link, pre_existing', [
+    "setting, expect_link, pre_existing",
+    [
         (None, True, None),
         (True, False, None),
         (False, True, None),
-        (True, False, 'link'),
-        (True, False, 'file'),
+        (True, False, "link"),
+        (True, False, "file"),
     ],
     ids=[
-        'unset',
-        'true',
-        'false',
-        'pre-existing symlink',
-        'pre-existing file',
-    ])
-@pytest.mark.usefixtures('ds1_copy')
-def test_alt_copy(
-        runner, yadm_y, paths, tst_sys,
-        setting, expect_link, pre_existing,
-        compatibility, cygwin):
+        "unset",
+        "true",
+        "false",
+        "pre-existing symlink",
+        "pre-existing file",
+    ],
+)
+@pytest.mark.usefixtures("ds1_copy")
+def test_alt_copy(runner, yadm_cmd, paths, tst_sys, setting, expect_link, pre_existing):
     """Test yadm.alt-copy"""
 
-    option = 'yadm.cygwin-copy' if cygwin else 'yadm.alt-copy'
-
     if setting is not None:
-        os.system(' '.join(yadm_y('config', option, str(setting))))
+        os.system(" ".join(yadm_cmd("config", "yadm.alt-copy", str(setting))))
 
-    if compatibility:
-        expected_content = f'test_alt_copy##{tst_sys}'
-    else:
-        expected_content = f'test_alt_copy##os.{tst_sys}'
+    expected_content = f"test_alt_copy##os.{tst_sys}"
 
-    alt_path = paths.work.join('test_alt_copy')
-    if pre_existing == 'symlink':
+    alt_path = paths.work.join("test_alt_copy")
+    if pre_existing == "symlink":
         alt_path.mklinkto(expected_content)
-    elif pre_existing == 'file':
-        alt_path.write('wrong content')
+    elif pre_existing == "file":
+        alt_path.write("wrong content")
 
-    env = os.environ.copy()
-    if compatibility:
-        env['YADM_COMPATIBILITY'] = '1'
-
-    run = runner(yadm_y('alt'), env=env)
+    run = runner(yadm_cmd("alt"))
     assert run.success
-    assert run.err == ''
-    assert 'Linking' in run.out
+    assert run.err == ""
+    assert "Linking" in run.out
 
     assert alt_path.read() == expected_content
     assert alt_path.islink() == expect_link

test/test_assert_private_dirs.py

@@ -2,13 +2,15 @@
 
 import os
 import re
+
 import pytest
 
-pytestmark = pytest.mark.usefixtures('ds1_copy')
-PRIVATE_DIRS = ['.gnupg', '.ssh']
+pytestmark = pytest.mark.usefixtures("ds1_copy")
+PRIVATE_DIRS = [".gnupg", ".ssh"]
 
 
-def test_pdirs_missing(runner, yadm_y, paths):
+@pytest.mark.parametrize("home", [True, False], ids=["home", "not-home"])
+def test_pdirs_missing(runner, yadm_cmd, paths, home):
     """Private dirs (private dirs missing)
 
     When a git command is run
@@ -23,26 +25,34 @@ def test_pdirs_missing(runner, yadm_y, paths):
             path.remove()
         assert not path.exists()
 
+    env = {"DEBUG": "yes"}
+    if home:
+        env["HOME"] = paths.work
+
     # run status
-    run = runner(command=yadm_y('status'), env={'DEBUG': 'yes'})
+    run = runner(command=yadm_cmd("status"), env=env)
     assert run.success
-    assert run.err == ''
-    assert 'On branch master' in run.out
+    assert run.err == ""
+    assert "On branch master" in run.out
 
     # confirm directories are created
     # and are protected
     for pdir in PRIVATE_DIRS:
         path = paths.work.join(pdir)
-        assert path.exists()
-        assert oct(path.stat().mode).endswith('00'), 'Directory is not secured'
+        if home:
+            assert path.exists()
+            assert oct(path.stat().mode).endswith("00"), "Directory is " "not secured"
+        else:
+            assert not path.exists()
 
     # confirm directories are created before command is run:
-    assert re.search(
-        r'Creating.+\.gnupg.+Creating.+\.ssh.+Running git command git status',
-        run.out, re.DOTALL), 'directories created before command is run'
+    if home:
+        assert re.search(
+            r"Creating.+\.(gnupg|ssh).+Creating.+\.(gnupg|ssh).+Running git command git status", run.out, re.DOTALL
+        ), "directories created before command is run"
 
 
-def test_pdirs_missing_apd_false(runner, yadm_y, paths):
+def test_pdirs_missing_apd_false(runner, yadm_cmd, paths):
     """Private dirs (private dirs missing / yadm.auto-private-dirs=false)
 
     When a git command is run
@@ -59,21 +69,20 @@ def test_pdirs_missing_apd_false(runner, yadm_y, paths):
         assert not path.exists()
 
     # set configuration
-    os.system(' '.join(yadm_y(
-        'config', '--bool', 'yadm.auto-private-dirs', 'false')))
+    os.system(" ".join(yadm_cmd("config", "--bool", "yadm.auto-private-dirs", "false")))
 
     # run status
-    run = runner(command=yadm_y('status'))
+    run = runner(command=yadm_cmd("status"))
     assert run.success
-    assert run.err == ''
-    assert 'On branch master' in run.out
+    assert run.err == ""
+    assert "On branch master" in run.out
 
     # confirm directories are STILL missing
     for pdir in PRIVATE_DIRS:
         assert not paths.work.join(pdir).exists()
 
 
-def test_pdirs_exist_apd_false(runner, yadm_y, paths):
+def test_pdirs_exist_apd_false(runner, yadm_cmd, paths):
     """Private dirs (private dirs exist / yadm.auto-perms=false)
 
     When a git command is run
@@ -88,19 +97,18 @@ def test_pdirs_exist_apd_false(runner, yadm_y, paths):
         if not path.isdir():
             path.mkdir()
         path.chmod(0o777)
-        assert oct(path.stat().mode).endswith('77'), 'Directory is secure.'
+        assert oct(path.stat().mode).endswith("77"), "Directory is secure."
 
     # set configuration
-    os.system(' '.join(yadm_y(
-        'config', '--bool', 'yadm.auto-perms', 'false')))
+    os.system(" ".join(yadm_cmd("config", "--bool", "yadm.auto-perms", "false")))
 
     # run status
-    run = runner(command=yadm_y('status'))
+    run = runner(command=yadm_cmd("status"))
     assert run.success
-    assert run.err == ''
-    assert 'On branch master' in run.out
+    assert run.err == ""
+    assert "On branch master" in run.out
 
     # created directories are STILL permissive
     for pdir in PRIVATE_DIRS:
         path = paths.work.join(pdir)
-        assert oct(path.stat().mode).endswith('77'), 'Directory is secure'
+        assert oct(path.stat().mode).endswith("77"), "Directory is secure"

test/test_bootstrap.py

@@ -4,28 +4,30 @@ import pytest
 
 
 @pytest.mark.parametrize(
-    'exists, executable, code, expect', [
-        (False, False, 1, 'Cannot execute bootstrap'),
-        (True, False, 1, 'is not an executable program'),
-        (True, True, 123, 'Bootstrap successful'),
-    ], ids=[
-        'missing',
-        'not executable',
-        'executable',
-    ])
-def test_bootstrap(
-        runner, yadm_y, paths, exists, executable, code, expect):
+    "exists, executable, code, expect",
+    [
+        (False, False, 1, "Cannot execute bootstrap"),
+        (True, False, 1, "is not an executable program"),
+        (True, True, 123, "Bootstrap successful"),
+    ],
+    ids=[
+        "missing",
+        "not executable",
+        "executable",
+    ],
+)
+def test_bootstrap(runner, yadm_cmd, paths, exists, executable, code, expect):
     """Test bootstrap command"""
     if exists:
-        paths.bootstrap.write('')
+        paths.bootstrap.write("")
     if executable:
-        paths.bootstrap.write(
-            '#!/bin/bash\n'
-            f'echo {expect}\n'
-            f'exit {code}\n'
-        )
+        paths.bootstrap.write("#!/bin/bash\n" f"echo {expect}\n" f"exit {code}\n")
         paths.bootstrap.chmod(0o775)
-    run = runner(command=yadm_y('bootstrap'))
+    run = runner(command=yadm_cmd("bootstrap"))
     assert run.code == code
-    assert run.err == ''
-    assert expect in run.out
+    if exists and executable:
+        assert run.err == ""
+        assert expect in run.out
+    else:
+        assert expect in run.err
+        assert run.out == ""

test/test_clean.py

@@ -1,11 +1,11 @@
 """Test clean"""
 
 
-def test_clean_command(runner, yadm_y):
+def test_clean_command(runner, yadm_cmd):
     """Run with clean command"""
-    run = runner(command=yadm_y('clean'))
+    run = runner(command=yadm_cmd("clean"))
     # do nothing, this is a dangerous Git command when managing dot files
     # report the command as disabled and exit as a failure
     assert run.failure
-    assert run.err == ''
-    assert 'disabled' in run.out
+    assert run.out == ""
+    assert "disabled" in run.err

test/test_clone.py

@@ -2,30 +2,32 @@
 
 import os
 import re
+
 import pytest
 
 BOOTSTRAP_CODE = 123
-BOOTSTRAP_MSG = 'Bootstrap successful'
+BOOTSTRAP_MSG = "Bootstrap successful"
 
 
-@pytest.mark.usefixtures('remote')
+@pytest.mark.usefixtures("remote")
 @pytest.mark.parametrize(
-    'good_remote, repo_exists, force, conflicts', [
+    "good_remote, repo_exists, force, conflicts",
+    [
         (False, False, False, False),
         (True, False, False, False),
         (True, True, False, False),
         (True, True, True, False),
         (True, False, False, True),
-    ], ids=[
-        'bad remote',
-        'simple',
-        'existing repo',
-        '-f',
-        'conflicts',
-    ])
-def test_clone(
-        runner, paths, yadm_y, repo_config, ds1,
-        good_remote, repo_exists, force, conflicts):
+    ],
+    ids=[
+        "bad remote",
+        "simple",
+        "existing repo",
+        "-f",
+        "conflicts",
+    ],
+)
+def test_clone(runner, paths, yadm_cmd, repo_config, ds1, good_remote, repo_exists, force, conflicts):
     """Test basic clone operation"""
 
     # clear out the work path
@@ -33,78 +35,70 @@ def test_clone(
     paths.work.mkdir()
 
     # determine remote url
-    remote_url = f'file://{paths.remote}'
+    remote_url = f"file://{paths.remote}"
     if not good_remote:
-        remote_url = 'file://bad_remote'
+        remote_url = "file://bad_remote"
 
     old_repo = None
     if repo_exists:
         # put a repo in the way
         paths.repo.mkdir()
-        old_repo = paths.repo.join('old_repo')
-        old_repo.write('old_repo')
+        old_repo = paths.repo.join("old_repo")
+        old_repo.write("old_repo")
 
     if conflicts:
-        ds1.tracked[0].relative.write('conflict')
+        ds1.tracked[0].relative.write("conflict")
         assert ds1.tracked[0].relative.exists()
 
     # run the clone command
-    args = ['clone', '-w', paths.work]
+    args = ["clone", "-w", paths.work]
     if force:
-        args += ['-f']
+        args += ["-f"]
     args += [remote_url]
-    run = runner(command=yadm_y(*args))
+    run = runner(command=yadm_cmd(*args))
 
     if not good_remote:
         # clone should fail
         assert run.failure
-        assert run.err != ''
-        assert 'Unable to fetch origin' in run.out
+        assert run.out == ""
+        assert "Unable to clone the repository" in run.err
         assert not paths.repo.exists()
     elif repo_exists and not force:
         # can't overwrite data
         assert run.failure
-        assert run.err == ''
-        assert 'Git repo already exists' in run.out
+        assert run.out == ""
+        assert "Git repo already exists" in run.err
     else:
         # clone should succeed, and repo should be configured properly
         assert successful_clone(run, paths, repo_config)
 
+        # these clones should have master as HEAD
+        verify_head(paths, "master")
+
         # ensure conflicts are handled properly
         if conflicts:
-            assert 'NOTE' in run.out
-            assert 'Merging origin/master failed' in run.out
-            assert 'Conflicts preserved' in run.out
+            assert "NOTE" in run.out
+            assert "Local files with content that differs" in run.out
 
         # confirm correct Git origin
-        run = runner(
-            command=('git', 'remote', '-v', 'show'),
-            env={'GIT_DIR': paths.repo})
+        run = runner(command=("git", "remote", "-v", "show"), env={"GIT_DIR": paths.repo})
         assert run.success
-        assert run.err == ''
-        assert f'origin\t{remote_url}' in run.out
+        assert run.err == ""
+        assert f"origin\t{remote_url}" in run.out
 
         # ensure conflicts are really preserved
         if conflicts:
-            # test to see if the work tree is actually "clean"
-            run = runner(
-                command=yadm_y('status', '-uno', '--porcelain'),
-                cwd=paths.work)
+            # test that the conflicts are preserved in the work tree
+            run = runner(command=yadm_cmd("status", "-uno", "--porcelain"), cwd=paths.work)
             assert run.success
-            assert run.err == ''
-            assert run.out == '', 'worktree has unexpected changes'
+            assert run.err == ""
+            assert str(ds1.tracked[0].path) in run.out
 
-            # test to see if the conflicts are stashed
-            run = runner(command=yadm_y('stash', 'list'), cwd=paths.work)
+            # verify content of the conflicts
+            run = runner(command=yadm_cmd("diff"), cwd=paths.work)
             assert run.success
-            assert run.err == ''
-            assert 'Conflicts preserved' in run.out, 'conflicts not stashed'
-
-            # verify content of the stashed conflicts
-            run = runner(command=yadm_y('stash', 'show', '-p'), cwd=paths.work)
-            assert run.success
-            assert run.err == ''
-            assert '\n+conflict' in run.out, 'conflicts not stashed'
+            assert run.err == ""
+            assert "\n+conflict" in run.out, "conflict overwritten"
 
     # another force-related assertion
     if old_repo:
@@ -114,53 +108,56 @@ def test_clone(
             assert old_repo.exists()
 
 
-@pytest.mark.usefixtures('remote')
+@pytest.mark.usefixtures("remote")
 @pytest.mark.parametrize(
-    'bs_exists, bs_param, answer', [
-        (False, '--bootstrap', None),
-        (True, '--bootstrap', None),
-        (True, '--no-bootstrap', None),
-        (True, None, 'n'),
-        (True, None, 'y'),
-    ], ids=[
-        'force, missing',
-        'force, existing',
-        'prevent',
-        'existing, answer n',
-        'existing, answer y',
-    ])
-def test_clone_bootstrap(
-        runner, paths, yadm_y, repo_config, bs_exists, bs_param, answer):
+    "bs_exists, bs_param, answer",
+    [
+        (False, "--bootstrap", None),
+        (True, "--bootstrap", None),
+        (True, "--no-bootstrap", None),
+        (True, None, "n"),
+        (True, None, "y"),
+    ],
+    ids=[
+        "force, missing",
+        "force, existing",
+        "prevent",
+        "existing, answer n",
+        "existing, answer y",
+    ],
+)
+def test_clone_bootstrap(runner, paths, yadm_cmd, repo_config, bs_exists, bs_param, answer):
     """Test bootstrap clone features"""
 
     # establish a bootstrap
     create_bootstrap(paths, bs_exists)
 
     # run the clone command
-    args = ['clone', '-w', paths.work]
+    args = ["clone", "-w", paths.work]
     if bs_param:
         args += [bs_param]
-    args += [f'file://{paths.remote}']
+    args += [f"file://{paths.remote}"]
     expect = []
     if answer:
-        expect.append(('Would you like to execute it now', answer))
-    run = runner(command=yadm_y(*args), expect=expect)
+        expect.append(("Would you like to execute it now", answer))
+    run = runner(command=yadm_cmd(*args), expect=expect)
 
     if answer:
-        assert 'Would you like to execute it now' in run.out
+        assert "Would you like to execute it now" in run.out
 
     expected_code = 0
-    if bs_exists and bs_param != '--no-bootstrap':
+    if bs_exists and bs_param != "--no-bootstrap":
         expected_code = BOOTSTRAP_CODE
 
-    if answer == 'y':
+    if answer == "y":
         expected_code = BOOTSTRAP_CODE
         assert BOOTSTRAP_MSG in run.out
-    elif answer == 'n':
+    elif answer == "n":
         expected_code = 0
         assert BOOTSTRAP_MSG not in run.out
 
     assert successful_clone(run, paths, repo_config, expected_code)
+    verify_head(paths, "master")
 
     if not bs_exists:
         assert BOOTSTRAP_MSG not in run.out
@@ -169,145 +166,137 @@ def test_clone_bootstrap(
 def create_bootstrap(paths, exists):
     """Create bootstrap file for test"""
     if exists:
-        paths.bootstrap.write(
-            '#!/bin/sh\n'
-            f'echo {BOOTSTRAP_MSG}\n'
-            f'exit {BOOTSTRAP_CODE}\n')
+        paths.bootstrap.write("#!/bin/sh\n" f"echo {BOOTSTRAP_MSG}\n" f"exit {BOOTSTRAP_CODE}\n")
         paths.bootstrap.chmod(0o775)
         assert paths.bootstrap.exists()
     else:
         assert not paths.bootstrap.exists()
 
 
-@pytest.mark.usefixtures('remote')
+@pytest.mark.usefixtures("remote")
 @pytest.mark.parametrize(
-    'private_type, in_repo, in_work', [
-        ('ssh', False, True),
-        ('gnupg', False, True),
-        ('ssh', True, True),
-        ('gnupg', True, True),
-        ('ssh', True, False),
-        ('gnupg', True, False),
-    ], ids=[
-        'open ssh, not tracked',
-        'open gnupg, not tracked',
-        'open ssh, tracked',
-        'open gnupg, tracked',
-        'missing ssh, tracked',
-        'missing gnupg, tracked',
-    ])
-def test_clone_perms(
-        runner, yadm_y, paths, repo_config,
-        private_type, in_repo, in_work):
+    "private_type, in_repo, in_work",
+    [
+        ("ssh", False, True),
+        ("gnupg", False, True),
+        ("ssh", True, True),
+        ("gnupg", True, True),
+        ("ssh", True, False),
+        ("gnupg", True, False),
+    ],
+    ids=[
+        "open ssh, not tracked",
+        "open gnupg, not tracked",
+        "open ssh, tracked",
+        "open gnupg, tracked",
+        "missing ssh, tracked",
+        "missing gnupg, tracked",
+    ],
+)
+def test_clone_perms(runner, yadm_cmd, paths, repo_config, private_type, in_repo, in_work):
     """Test clone permission-related functions"""
 
     # update remote repo to include private data
     if in_repo:
-        rpath = paths.work.mkdir(f'.{private_type}').join('related')
-        rpath.write('related')
+        rpath = paths.work.mkdir(f".{private_type}").join("related")
+        rpath.write("related")
         os.system(f'GIT_DIR="{paths.remote}" git add {rpath}')
         os.system(f'GIT_DIR="{paths.remote}" git commit -m "{rpath}"')
         rpath.remove()
 
     # ensure local private data is insecure at the start
     if in_work:
-        pdir = paths.work.join(f'.{private_type}')
+        pdir = paths.work.join(f".{private_type}")
         if not pdir.exists():
             pdir.mkdir()
-        pfile = pdir.join('existing')
-        pfile.write('existing')
+        pfile = pdir.join("existing")
+        pfile.write("existing")
         pdir.chmod(0o777)
         pfile.chmod(0o777)
     else:
         paths.work.remove()
         paths.work.mkdir()
 
-    run = runner(
-        yadm_y('clone', '-d', '-w', paths.work, f'file://{paths.remote}'))
+    env = {"HOME": paths.work}
+    run = runner(yadm_cmd("clone", "-d", "-w", paths.work, f"file://{paths.remote}"), env=env)
 
     assert successful_clone(run, paths, repo_config)
+    verify_head(paths, "master")
     if in_work:
         # private directories which already exist, should be left as they are,
         # which in this test is "insecure".
-        assert re.search(
-            f'initial private dir perms drwxrwxrwx.+.{private_type}',
-            run.out)
-        assert re.search(
-            f'pre-merge private dir perms drwxrwxrwx.+.{private_type}',
-            run.out)
-        assert re.search(
-            f'post-merge private dir perms drwxrwxrwx.+.{private_type}',
-            run.out)
+        assert re.search(f"initial private dir perms drwxrwxrwx.+.{private_type}", run.out)
+        assert re.search(f"pre-checkout private dir perms drwxrwxrwx.+.{private_type}", run.out)
+        assert re.search(f"post-checkout private dir perms drwxrwxrwx.+.{private_type}", run.out)
     else:
         # private directories which are created, should be done prior to
-        # merging, and with secure permissions.
-        assert 'initial private dir perms' not in run.out
-        assert re.search(
-            f'pre-merge private dir perms drwx------.+.{private_type}',
-            run.out)
-        assert re.search(
-            f'post-merge private dir perms drwx------.+.{private_type}',
-            run.out)
+        # checkout, and with secure permissions.
+        assert "initial private dir perms" not in run.out
+        assert re.search(f"pre-checkout private dir perms drwx------.+.{private_type}", run.out)
+        assert re.search(f"post-checkout private dir perms drwx------.+.{private_type}", run.out)
 
     # standard perms still apply afterwards unless disabled with auto.perms
-    assert oct(
-        paths.work.join(f'.{private_type}').stat().mode).endswith('00'), (
-            f'.{private_type} has not been secured by auto.perms')
+    assert oct(paths.work.join(f".{private_type}").stat().mode).endswith(
+        "00"
+    ), f".{private_type} has not been secured by auto.perms"
 
 
-@pytest.mark.usefixtures('remote')
-@pytest.mark.parametrize('branch', ['master', 'valid', 'invalid'])
-def test_alternate_branch(runner, paths, yadm_y, repo_config, branch):
+@pytest.mark.usefixtures("remote")
+@pytest.mark.parametrize("branch", ["master", "default", "valid", "invalid"])
+def test_alternate_branch(runner, paths, yadm_cmd, repo_config, branch):
     """Test cloning a branch other than master"""
 
     # add a "valid" branch to the remote
     os.system(f'GIT_DIR="{paths.remote}" git checkout -b valid')
-    os.system(
-        f'GIT_DIR="{paths.remote}" git commit '
-        f'--allow-empty -m "This branch is valid"')
+    os.system(f'GIT_DIR="{paths.remote}" git commit ' f'--allow-empty -m "This branch is valid"')
+    if branch != "default":
+        # When branch == 'default', the "default" branch of the remote repo
+        # will remain "valid" to validate identification the correct default
+        # branch by inspecting the repo. Otherwise it will be set back to
+        # "master"
+        os.system(f'GIT_DIR="{paths.remote}" git checkout master')
 
     # clear out the work path
     paths.work.remove()
     paths.work.mkdir()
 
-    remote_url = f'file://{paths.remote}'
+    remote_url = f"file://{paths.remote}"
 
     # run the clone command
-    args = ['clone', '-w', paths.work]
-    if branch != 'master':
-        args += ['-b', branch]
+    args = ["clone", "-w", paths.work]
+    if branch not in ["master", "default"]:
+        args += ["-b", branch]
     args += [remote_url]
-    run = runner(command=yadm_y(*args))
+    run = runner(command=yadm_cmd(*args))
 
-    if branch == 'invalid':
+    if branch == "invalid":
         assert run.failure
-        assert 'ERROR: Clone failed' in run.out
-        assert f"'origin/{branch}' does not exist in {remote_url}" in run.out
+        assert "ERROR: Unable to clone the repository" in run.err
+        assert f"Remote branch {branch} not found in upstream" in run.err
     else:
         assert successful_clone(run, paths, repo_config)
 
         # confirm correct Git origin
-        run = runner(
-            command=('git', 'remote', '-v', 'show'),
-            env={'GIT_DIR': paths.repo})
+        run = runner(command=("git", "remote", "-v", "show"), env={"GIT_DIR": paths.repo})
         assert run.success
-        assert run.err == ''
-        assert f'origin\t{remote_url}' in run.out
-        run = runner(command=yadm_y('show'))
-        if branch == 'valid':
-            assert 'This branch is valid' in run.out
+        assert run.err == ""
+        assert f"origin\t{remote_url}" in run.out
+        run = runner(command=yadm_cmd("show"))
+        if branch == "master":
+            assert "Initial commit" in run.out
+            verify_head(paths, "master")
         else:
-            assert 'Initial commit' in run.out
+            assert "This branch is valid" in run.out
+            verify_head(paths, "valid")
 
 
 def successful_clone(run, paths, repo_config, expected_code=0):
     """Assert clone is successful"""
     assert run.code == expected_code
-    assert 'Initialized' in run.out
-    assert oct(paths.repo.stat().mode).endswith('00'), 'Repo is not secured'
-    assert repo_config('core.bare') == 'false'
-    assert repo_config('status.showUntrackedFiles') == 'no'
-    assert repo_config('yadm.managed') == 'true'
+    assert oct(paths.repo.stat().mode).endswith("00"), "Repo is not secured"
+    assert repo_config("core.bare") == "false"
+    assert repo_config("status.showUntrackedFiles") == "no"
+    assert repo_config("yadm.managed") == "true"
     return True
 
 
@@ -320,3 +309,20 @@ def remote(paths, ds1_repo_copy):
     # cannot be applied to another fixture.
     paths.remote.remove()
     paths.repo.move(paths.remote)
+
+
+def test_no_repo(
+    runner,
+    yadm_cmd,
+):
+    """Test cloning without specifying a repo"""
+    run = runner(command=yadm_cmd("clone", "-f"))
+    assert run.failure
+    assert run.out == ""
+    assert "ERROR: Unable to clone the repository" in run.err
+    assert "repository 'repo.git' does not exist" in run.err
+
+
+def verify_head(paths, branch):
+    """Assert the local repo has the correct head branch"""
+    assert paths.repo.join("HEAD").read() == f"ref: refs/heads/{branch}\n"

test/test_compat_alt.py

@@ -1,450 +0,0 @@
-"""Test alt"""
-
-import os
-import string
-import py
-import pytest
-import utils
-
-# These tests are for the alternate processing in YADM_COMPATIBILITY=1 mode
-pytestmark = pytest.mark.deprecated
-
-# These test IDs are broken. During the writing of these tests, problems have
-# been discovered in the way yadm orders matching files.
-BROKEN_TEST_IDS = [
-    'test_wild[tracked-##C.S.H.U-C-S%-H%-U]',
-    'test_wild[tracked-##C.S.H.U-C-S-H%-U]',
-    'test_wild[encrypted-##C.S.H.U-C-S%-H%-U]',
-    'test_wild[encrypted-##C.S.H.U-C-S-H%-U]',
-    ]
-
-PRECEDENCE = [
-    '##',
-    '##$tst_sys',
-    '##$tst_sys.$tst_host',
-    '##$tst_sys.$tst_host.$tst_user',
-    '##$tst_class',
-    '##$tst_class.$tst_sys',
-    '##$tst_class.$tst_sys.$tst_host',
-    '##$tst_class.$tst_sys.$tst_host.$tst_user',
-    ]
-
-WILD_TEMPLATES = [
-    '##$tst_class',
-    '##$tst_class.$tst_sys',
-    '##$tst_class.$tst_sys.$tst_host',
-    '##$tst_class.$tst_sys.$tst_host.$tst_user',
-    ]
-
-TEST_PATHS = [utils.ALT_FILE1, utils.ALT_FILE2, utils.ALT_DIR]
-
-WILD_TESTED = set()
-
-
-@pytest.mark.parametrize('precedence_index', range(len(PRECEDENCE)))
-@pytest.mark.parametrize(
-    'tracked, encrypt, exclude', [
-        (False, False, False),
-        (True, False, False),
-        (False, True, False),
-        (False, True, True),
-    ], ids=[
-        'untracked',
-        'tracked',
-        'encrypted',
-        'excluded',
-    ])
-@pytest.mark.usefixtures('ds1_copy')
-def test_alt(runner, yadm_y, paths,
-             tst_sys, tst_host, tst_user,
-             tracked, encrypt, exclude,
-             precedence_index):
-    """Test alternate linking
-
-    This test is done by iterating for the number of templates in PRECEDENCE.
-    With each iteration, another file is left off the list. So with each
-    iteration, the template with the "highest precedence" is left out. The file
-    using the highest precedence should be the one linked.
-    """
-
-    # set the class
-    tst_class = 'testclass'
-    utils.set_local(paths, 'class', tst_class)
-
-    # process the templates in PRECEDENCE
-    precedence = list()
-    for template in PRECEDENCE:
-        precedence.append(
-            string.Template(template).substitute(
-                tst_class=tst_class,
-                tst_host=tst_host,
-                tst_sys=tst_sys,
-                tst_user=tst_user,
-            )
-        )
-
-    # create files using a subset of files
-    for suffix in precedence[0:precedence_index+1]:
-        utils.create_alt_files(paths, suffix, tracked=tracked,
-                               encrypt=encrypt, exclude=exclude)
-
-    # run alt to trigger linking
-    env = os.environ.copy()
-    env['YADM_COMPATIBILITY'] = '1'
-    run = runner(yadm_y('alt'), env=env)
-    assert run.success
-    assert run.err == ''
-    linked = utils.parse_alt_output(run.out)
-
-    # assert the proper linking has occurred
-    for file_path in TEST_PATHS:
-        source_file = file_path + precedence[precedence_index]
-        if tracked or (encrypt and not exclude):
-            assert paths.work.join(file_path).islink()
-            target = py.path.local(paths.work.join(file_path).readlink())
-            if target.isfile():
-                assert paths.work.join(file_path).read() == source_file
-                assert str(paths.work.join(source_file)) in linked
-            else:
-                assert paths.work.join(file_path).join(
-                    utils.CONTAINED).read() == source_file
-                assert str(paths.work.join(source_file)) in linked
-        else:
-            assert not paths.work.join(file_path).exists()
-            assert str(paths.work.join(source_file)) not in linked
-
-
-def short_template(template):
-    """Translate template into something short for test IDs"""
-    return string.Template(template).substitute(
-        tst_class='C',
-        tst_host='H',
-        tst_sys='S',
-        tst_user='U',
-    )
-
-
-@pytest.mark.parametrize('wild_user', [True, False], ids=['U%', 'U'])
-@pytest.mark.parametrize('wild_host', [True, False], ids=['H%', 'H'])
-@pytest.mark.parametrize('wild_sys', [True, False], ids=['S%', 'S'])
-@pytest.mark.parametrize('wild_class', [True, False], ids=['C%', 'C'])
-@pytest.mark.parametrize('template', WILD_TEMPLATES, ids=short_template)
-@pytest.mark.parametrize(
-    'tracked, encrypt', [
-        (True, False),
-        (False, True),
-    ], ids=[
-        'tracked',
-        'encrypted',
-    ])
-@pytest.mark.usefixtures('ds1_copy')
-def test_wild(request, runner, yadm_y, paths,
-              tst_sys, tst_host, tst_user,
-              tracked, encrypt,
-              wild_class, wild_host, wild_sys, wild_user,
-              template):
-    """Test wild linking
-
-    These tests are done by creating permutations of the possible files using
-    WILD_TEMPLATES. Each case is then tested (while skipping the already tested
-    permutations for efficiency).
-    """
-
-    if request.node.name in BROKEN_TEST_IDS:
-        pytest.xfail(
-            'This test is known to be broken. '
-            'This bug only affects deprecated features.')
-
-    tst_class = 'testclass'
-
-    # determine the "wild" version of the suffix
-    str_class = '%' if wild_class else tst_class
-    str_host = '%' if wild_host else tst_host
-    str_sys = '%' if wild_sys else tst_sys
-    str_user = '%' if wild_user else tst_user
-    wild_suffix = string.Template(template).substitute(
-        tst_class=str_class,
-        tst_host=str_host,
-        tst_sys=str_sys,
-        tst_user=str_user,
-    )
-
-    # determine the "standard" version of the suffix
-    std_suffix = string.Template(template).substitute(
-        tst_class=tst_class,
-        tst_host=tst_host,
-        tst_sys=tst_sys,
-        tst_user=tst_user,
-    )
-
-    # skip over duplicate tests (this seems to be the simplest way to cover the
-    # permutations of tests, while skipping duplicates.)
-    test_key = f'{tracked}{encrypt}{wild_suffix}{std_suffix}'
-    if test_key in WILD_TESTED:
-        return
-    WILD_TESTED.add(test_key)
-
-    # set the class
-    utils.set_local(paths, 'class', tst_class)
-
-    # create files using the wild suffix
-    utils.create_alt_files(paths, wild_suffix, tracked=tracked,
-                           encrypt=encrypt, exclude=False)
-
-    # run alt to trigger linking
-    env = os.environ.copy()
-    env['YADM_COMPATIBILITY'] = '1'
-    run = runner(yadm_y('alt'), env=env)
-    assert run.success
-    assert run.err == ''
-    linked = utils.parse_alt_output(run.out)
-
-    # assert the proper linking has occurred
-    for file_path in TEST_PATHS:
-        source_file = file_path + wild_suffix
-        assert paths.work.join(file_path).islink()
-        target = py.path.local(paths.work.join(file_path).readlink())
-        if target.isfile():
-            assert paths.work.join(file_path).read() == source_file
-            assert str(paths.work.join(source_file)) in linked
-        else:
-            assert paths.work.join(file_path).join(
-                utils.CONTAINED).read() == source_file
-            assert str(paths.work.join(source_file)) in linked
-
-    # create files using the standard suffix
-    utils.create_alt_files(paths, std_suffix, tracked=tracked,
-                           encrypt=encrypt, exclude=False)
-
-    # run alt to trigger linking
-    env = os.environ.copy()
-    env['YADM_COMPATIBILITY'] = '1'
-    run = runner(yadm_y('alt'), env=env)
-    assert run.success
-    assert run.err == ''
-    linked = utils.parse_alt_output(run.out)
-
-    # assert the proper linking has occurred
-    for file_path in TEST_PATHS:
-        source_file = file_path + std_suffix
-        assert paths.work.join(file_path).islink()
-        target = py.path.local(paths.work.join(file_path).readlink())
-        if target.isfile():
-            assert paths.work.join(file_path).read() == source_file
-            assert str(paths.work.join(source_file)) in linked
-        else:
-            assert paths.work.join(file_path).join(
-                utils.CONTAINED).read() == source_file
-            assert str(paths.work.join(source_file)) in linked
-
-
-@pytest.mark.usefixtures('ds1_copy')
-def test_local_override(runner, yadm_y, paths,
-                        tst_sys, tst_host, tst_user):
-    """Test local overrides"""
-
-    # define local overrides
-    utils.set_local(paths, 'class', 'or-class')
-    utils.set_local(paths, 'hostname', 'or-hostname')
-    utils.set_local(paths, 'os', 'or-os')
-    utils.set_local(paths, 'user', 'or-user')
-
-    # create files, the first would normally be the most specific version
-    # however, the second is the overridden version which should be preferred.
-    utils.create_alt_files(
-        paths, f'##or-class.{tst_sys}.{tst_host}.{tst_user}')
-    utils.create_alt_files(
-        paths, '##or-class.or-os.or-hostname.or-user')
-
-    # run alt to trigger linking
-    env = os.environ.copy()
-    env['YADM_COMPATIBILITY'] = '1'
-    run = runner(yadm_y('alt'), env=env)
-    assert run.success
-    assert run.err == ''
-    linked = utils.parse_alt_output(run.out)
-
-    # assert the proper linking has occurred
-    for file_path in TEST_PATHS:
-        source_file = file_path + '##or-class.or-os.or-hostname.or-user'
-        assert paths.work.join(file_path).islink()
-        target = py.path.local(paths.work.join(file_path).readlink())
-        if target.isfile():
-            assert paths.work.join(file_path).read() == source_file
-            assert str(paths.work.join(source_file)) in linked
-        else:
-            assert paths.work.join(file_path).join(
-                utils.CONTAINED).read() == source_file
-            assert str(paths.work.join(source_file)) in linked
-
-
-@pytest.mark.parametrize('suffix', ['AAA', 'ZZZ', 'aaa', 'zzz'])
-@pytest.mark.usefixtures('ds1_copy')
-def test_class_case(runner, yadm_y, paths, tst_sys, suffix):
-    """Test range of class cases"""
-
-    # set the class
-    utils.set_local(paths, 'class', suffix)
-
-    # create files
-    endings = [suffix]
-    if tst_sys == 'Linux':
-        # Only create all of these side-by-side on Linux, which is
-        # unquestionably case-sensitive. This would break tests on
-        # case-insensitive systems.
-        endings = ['AAA', 'ZZZ', 'aaa', 'zzz']
-    for ending in endings:
-        utils.create_alt_files(paths, f'##{ending}')
-
-    # run alt to trigger linking
-    env = os.environ.copy()
-    env['YADM_COMPATIBILITY'] = '1'
-    run = runner(yadm_y('alt'), env=env)
-    assert run.success
-    assert run.err == ''
-    linked = utils.parse_alt_output(run.out)
-
-    # assert the proper linking has occurred
-    for file_path in TEST_PATHS:
-        source_file = file_path + f'##{suffix}'
-        assert paths.work.join(file_path).islink()
-        target = py.path.local(paths.work.join(file_path).readlink())
-        if target.isfile():
-            assert paths.work.join(file_path).read() == source_file
-            assert str(paths.work.join(source_file)) in linked
-        else:
-            assert paths.work.join(file_path).join(
-                utils.CONTAINED).read() == source_file
-            assert str(paths.work.join(source_file)) in linked
-
-
-@pytest.mark.parametrize('autoalt', [None, 'true', 'false'])
-@pytest.mark.usefixtures('ds1_copy')
-def test_auto_alt(runner, yadm_y, paths, autoalt):
-    """Test setting auto-alt"""
-
-    # set the value of auto-alt
-    if autoalt:
-        os.system(' '.join(yadm_y('config', 'yadm.auto-alt', autoalt)))
-
-    # create file
-    suffix = '##'
-    utils.create_alt_files(paths, suffix)
-
-    # run status to possibly trigger linking
-    env = os.environ.copy()
-    env['YADM_COMPATIBILITY'] = '1'
-    run = runner(yadm_y('status'), env=env)
-    assert run.success
-    assert run.err == ''
-    linked = utils.parse_alt_output(run.out)
-
-    # assert the proper linking has occurred
-    for file_path in TEST_PATHS:
-        source_file = file_path + suffix
-        if autoalt == 'false':
-            assert not paths.work.join(file_path).exists()
-        else:
-            assert paths.work.join(file_path).islink()
-            target = py.path.local(paths.work.join(file_path).readlink())
-            if target.isfile():
-                assert paths.work.join(file_path).read() == source_file
-                # no linking output when run via auto-alt
-                assert str(paths.work.join(source_file)) not in linked
-            else:
-                assert paths.work.join(file_path).join(
-                    utils.CONTAINED).read() == source_file
-                # no linking output when run via auto-alt
-                assert str(paths.work.join(source_file)) not in linked
-
-
-@pytest.mark.parametrize('delimiter', ['.', '_'])
-@pytest.mark.usefixtures('ds1_copy')
-def test_delimiter(runner, yadm_y, paths,
-                   tst_sys, tst_host, tst_user, delimiter):
-    """Test delimiters used"""
-
-    suffix = '##' + delimiter.join([tst_sys, tst_host, tst_user])
-
-    # create file
-    utils.create_alt_files(paths, suffix)
-
-    # run alt to trigger linking
-    env = os.environ.copy()
-    env['YADM_COMPATIBILITY'] = '1'
-    run = runner(yadm_y('alt'), env=env)
-    assert run.success
-    assert run.err == ''
-    linked = utils.parse_alt_output(run.out)
-
-    # assert the proper linking has occurred
-    # only a delimiter of '.' is valid
-    for file_path in TEST_PATHS:
-        source_file = file_path + suffix
-        if delimiter == '.':
-            assert paths.work.join(file_path).islink()
-            target = py.path.local(paths.work.join(file_path).readlink())
-            if target.isfile():
-                assert paths.work.join(file_path).read() == source_file
-                assert str(paths.work.join(source_file)) in linked
-            else:
-                assert paths.work.join(file_path).join(
-                    utils.CONTAINED).read() == source_file
-                assert str(paths.work.join(source_file)) in linked
-        else:
-            assert not paths.work.join(file_path).exists()
-            assert str(paths.work.join(source_file)) not in linked
-
-
-@pytest.mark.usefixtures('ds1_copy')
-def test_invalid_links_removed(runner, yadm_y, paths):
-    """Links to invalid alternative files are removed
-
-    This test ensures that when an already linked alternative becomes invalid
-    due to a change in class, the alternate link is removed.
-    """
-
-    # set the class
-    tst_class = 'testclass'
-    utils.set_local(paths, 'class', tst_class)
-
-    # create files which match the test class
-    utils.create_alt_files(paths, f'##{tst_class}')
-
-    # run alt to trigger linking
-    env = os.environ.copy()
-    env['YADM_COMPATIBILITY'] = '1'
-    run = runner(yadm_y('alt'), env=env)
-    assert run.success
-    assert run.err == ''
-    linked = utils.parse_alt_output(run.out)
-
-    # assert the proper linking has occurred
-    for file_path in TEST_PATHS:
-        source_file = file_path + '##' + tst_class
-        assert paths.work.join(file_path).islink()
-        target = py.path.local(paths.work.join(file_path).readlink())
-        if target.isfile():
-            assert paths.work.join(file_path).read() == source_file
-            assert str(paths.work.join(source_file)) in linked
-        else:
-            assert paths.work.join(file_path).join(
-                utils.CONTAINED).read() == source_file
-            assert str(paths.work.join(source_file)) in linked
-
-    # change the class so there are no valid alternates
-    utils.set_local(paths, 'class', 'changedclass')
-
-    # run alt to trigger linking
-    env = os.environ.copy()
-    env['YADM_COMPATIBILITY'] = '1'
-    run = runner(yadm_y('alt'), env=env)
-    assert run.success
-    assert run.err == ''
-    linked = utils.parse_alt_output(run.out)
-
-    # assert the linking is removed
-    for file_path in TEST_PATHS:
-        source_file = file_path + '##' + tst_class
-        assert not paths.work.join(file_path).exists()
-        assert str(paths.work.join(source_file)) not in linked

test/test_compat_jinja.py

@@ -1,198 +0,0 @@
-"""Test jinja"""
-
-import os
-import pytest
-import utils
-
-# These tests are for the template processing in YADM_COMPATIBILITY=1 mode
-pytestmark = pytest.mark.deprecated
-
-
-@pytest.fixture(scope='module')
-def envtpl_present(runner):
-    """Is envtpl present and working?"""
-    try:
-        run = runner(command=['envtpl', '-h'])
-        if run.success:
-            return True
-    except OSError:
-        pass
-    return False
-
-
-@pytest.mark.usefixtures('ds1_copy')
-def test_local_override(runner, yadm_y, paths,
-                        tst_distro, envtpl_present):
-    """Test local overrides"""
-    if not envtpl_present:
-        pytest.skip('Unable to test without envtpl.')
-
-    # define local overrides
-    utils.set_local(paths, 'class', 'or-class')
-    utils.set_local(paths, 'hostname', 'or-hostname')
-    utils.set_local(paths, 'os', 'or-os')
-    utils.set_local(paths, 'user', 'or-user')
-
-    template = (
-        'j2-{{ YADM_CLASS }}-'
-        '{{ YADM_OS }}-{{ YADM_HOSTNAME }}-'
-        '{{ YADM_USER }}-{{ YADM_DISTRO }}'
-        '-{%- '
-        f"include '{utils.INCLUDE_FILE}'"
-        ' -%}'
-    )
-    expected = (
-        f'j2-or-class-or-os-or-hostname-or-user-{tst_distro}'
-        f'-{utils.INCLUDE_CONTENT}'
-    )
-
-    utils.create_alt_files(paths, '##yadm.j2', content=template,
-                           includefile=True)
-
-    # os.system(f'find {paths.work}' + ' -name *j2 -ls -exec cat \'{}\' ";"')
-    # os.system(f'find {paths.work}')
-    # run alt to trigger linking
-    env = os.environ.copy()
-    env['YADM_COMPATIBILITY'] = '1'
-    run = runner(yadm_y('alt'), env=env)
-    assert run.success
-    assert run.err == ''
-    created = utils.parse_alt_output(run.out, linked=False)
-
-    # assert the proper creation has occurred
-    for file_path in (utils.ALT_FILE1, utils.ALT_FILE2):
-        source_file = file_path + '##yadm.j2'
-        assert paths.work.join(file_path).isfile()
-        lines = paths.work.join(file_path).readlines(cr=False)
-        assert lines[0] == source_file
-        assert lines[1] == expected
-        assert str(paths.work.join(source_file)) in created
-
-
-@pytest.mark.parametrize('autoalt', [None, 'true', 'false'])
-@pytest.mark.usefixtures('ds1_copy')
-def test_auto_alt(runner, yadm_y, paths, autoalt, tst_sys,
-                  envtpl_present):
-    """Test setting auto-alt"""
-
-    if not envtpl_present:
-        pytest.skip('Unable to test without envtpl.')
-
-    # set the value of auto-alt
-    if autoalt:
-        os.system(' '.join(yadm_y('config', 'yadm.auto-alt', autoalt)))
-
-    # create file
-    jinja_suffix = '##yadm.j2'
-    utils.create_alt_files(paths, jinja_suffix, content='{{ YADM_OS }}')
-
-    # run status to possibly trigger linking
-    env = os.environ.copy()
-    env['YADM_COMPATIBILITY'] = '1'
-    run = runner(yadm_y('status'), env=env)
-    assert run.success
-    assert run.err == ''
-    created = utils.parse_alt_output(run.out, linked=False)
-
-    # assert the proper creation has occurred
-    for file_path in (utils.ALT_FILE1, utils.ALT_FILE2):
-        source_file = file_path + jinja_suffix
-        if autoalt == 'false':
-            assert not paths.work.join(file_path).exists()
-        else:
-            assert paths.work.join(file_path).isfile()
-            lines = paths.work.join(file_path).readlines(cr=False)
-            assert lines[0] == source_file
-            assert lines[1] == tst_sys
-            # no created output when run via auto-alt
-            assert str(paths.work.join(source_file)) not in created
-
-
-@pytest.mark.usefixtures('ds1_copy')
-def test_jinja_envtpl_missing(runner, paths):
-    """Test operation when envtpl is missing"""
-
-    script = f"""
-        YADM_TEST=1 source {paths.pgm}
-        process_global_args -Y "{paths.yadm}"
-        set_operating_system
-        configure_paths
-        YADM_COMPATIBILITY=1
-        ENVTPL_PROGRAM='envtpl_missing' main alt
-    """
-
-    utils.create_alt_files(paths, '##yadm.j2')
-
-    run = runner(command=['bash'], inp=script)
-    assert run.success
-    assert run.err == ''
-    assert f'envtpl not available, not creating' in run.out
-
-
-@pytest.mark.parametrize(
-    'tracked, encrypt, exclude', [
-        (False, False, False),
-        (True, False, False),
-        (False, True, False),
-        (False, True, True),
-    ], ids=[
-        'untracked',
-        'tracked',
-        'encrypted',
-        'excluded',
-    ])
-@pytest.mark.usefixtures('ds1_copy')
-def test_jinja(runner, yadm_y, paths,
-               tst_sys, tst_host, tst_user, tst_distro,
-               tracked, encrypt, exclude,
-               envtpl_present):
-    """Test jinja processing"""
-
-    if not envtpl_present:
-        pytest.skip('Unable to test without envtpl.')
-
-    jinja_suffix = '##yadm.j2'
-
-    # set the class
-    tst_class = 'testclass'
-    utils.set_local(paths, 'class', tst_class)
-
-    template = (
-        'j2-{{ YADM_CLASS }}-'
-        '{{ YADM_OS }}-{{ YADM_HOSTNAME }}-'
-        '{{ YADM_USER }}-{{ YADM_DISTRO }}'
-        '-{%- '
-        f"include '{utils.INCLUDE_FILE}'"
-        ' -%}'
-    )
-    expected = (
-        f'j2-{tst_class}-'
-        f'{tst_sys}-{tst_host}-'
-        f'{tst_user}-{tst_distro}'
-        f'-{utils.INCLUDE_CONTENT}'
-    )
-
-    utils.create_alt_files(paths, jinja_suffix, content=template,
-                           tracked=tracked, encrypt=encrypt, exclude=exclude,
-                           includefile=True)
-
-    # run alt to trigger linking
-    env = os.environ.copy()
-    env['YADM_COMPATIBILITY'] = '1'
-    run = runner(yadm_y('alt'), env=env)
-    assert run.success
-    assert run.err == ''
-    created = utils.parse_alt_output(run.out, linked=False)
-
-    # assert the proper creation has occurred
-    for file_path in (utils.ALT_FILE1, utils.ALT_FILE2):
-        source_file = file_path + jinja_suffix
-        if tracked or (encrypt and not exclude):
-            assert paths.work.join(file_path).isfile()
-            lines = paths.work.join(file_path).readlines(cr=False)
-            assert lines[0] == source_file
-            assert lines[1] == expected
-            assert str(paths.work.join(source_file)) in created
-        else:
-            assert not paths.work.join(file_path).exists()
-            assert str(paths.work.join(source_file)) not in created

test/test_config.py

@@ -1,16 +1,17 @@
 """Test config"""
 
 import os
+
 import pytest
 
-TEST_SECTION = 'test'
-TEST_ATTRIBUTE = 'attribute'
-TEST_KEY = f'{TEST_SECTION}.{TEST_ATTRIBUTE}'
-TEST_VALUE = 'testvalue'
-TEST_FILE = f'[{TEST_SECTION}]\n\t{TEST_ATTRIBUTE} = {TEST_VALUE}'
+TEST_SECTION = "test"
+TEST_ATTRIBUTE = "attribute"
+TEST_KEY = f"{TEST_SECTION}.{TEST_ATTRIBUTE}"
+TEST_VALUE = "testvalue"
+TEST_FILE = f"[{TEST_SECTION}]\n\t{TEST_ATTRIBUTE} = {TEST_VALUE}"
 
 
-def test_config_no_params(runner, yadm_y, supported_configs):
+def test_config_no_params(runner, yadm_cmd, supported_configs):
     """No parameters
 
     Display instructions
@@ -18,30 +19,30 @@ def test_config_no_params(runner, yadm_y, supported_configs):
     Exit with 0
     """
 
-    run = runner(yadm_y('config'))
+    run = runner(yadm_cmd("config"))
 
     assert run.success
-    assert run.err == ''
-    assert 'Please read the CONFIGURATION section' in run.out
+    assert run.err == ""
+    assert "Please read the CONFIGURATION section" in run.out
     for config in supported_configs:
         assert config in run.out
 
 
-def test_config_read_missing(runner, yadm_y):
+def test_config_read_missing(runner, yadm_cmd):
     """Read missing attribute
 
     Display an empty value
     Exit with 0
     """
 
-    run = runner(yadm_y('config', TEST_KEY))
+    run = runner(yadm_cmd("config", TEST_KEY))
 
     assert run.success
-    assert run.err == ''
-    assert run.out == ''
+    assert run.err == ""
+    assert run.out == ""
 
 
-def test_config_write(runner, yadm_y, paths):
+def test_config_write(runner, yadm_cmd, paths):
     """Write attribute
 
     Display no output
@@ -49,15 +50,15 @@ def test_config_write(runner, yadm_y, paths):
     Exit with 0
     """
 
-    run = runner(yadm_y('config', TEST_KEY, TEST_VALUE))
+    run = runner(yadm_cmd("config", TEST_KEY, TEST_VALUE))
 
     assert run.success
-    assert run.err == ''
-    assert run.out == ''
+    assert run.err == ""
+    assert run.out == ""
     assert paths.config.read().strip() == TEST_FILE
 
 
-def test_config_read(runner, yadm_y, paths):
+def test_config_read(runner, yadm_cmd, paths):
     """Read attribute
 
     Display value
@@ -65,14 +66,14 @@ def test_config_read(runner, yadm_y, paths):
     """
 
     paths.config.write(TEST_FILE)
-    run = runner(yadm_y('config', TEST_KEY))
+    run = runner(yadm_cmd("config", TEST_KEY))
 
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
     assert run.out.strip() == TEST_VALUE
 
 
-def test_config_update(runner, yadm_y, paths):
+def test_config_update(runner, yadm_cmd, paths):
     """Update attribute
 
     Display no output
@@ -82,17 +83,17 @@ def test_config_update(runner, yadm_y, paths):
 
     paths.config.write(TEST_FILE)
 
-    run = runner(yadm_y('config', TEST_KEY, TEST_VALUE + 'extra'))
+    run = runner(yadm_cmd("config", TEST_KEY, TEST_VALUE + "extra"))
 
     assert run.success
-    assert run.err == ''
-    assert run.out == ''
+    assert run.err == ""
+    assert run.out == ""
 
-    assert paths.config.read().strip() == TEST_FILE + 'extra'
+    assert paths.config.read().strip() == TEST_FILE + "extra"
 
 
-@pytest.mark.usefixtures('ds1_repo_copy')
-def test_config_local_read(runner, yadm_y, paths, supported_local_configs):
+@pytest.mark.usefixtures("ds1_repo_copy")
+def test_config_local_read(runner, yadm_cmd, paths, supported_local_configs):
     """Read local attribute
 
     Display value from the repo config
@@ -101,20 +102,18 @@ def test_config_local_read(runner, yadm_y, paths, supported_local_configs):
 
     # populate test values
     for config in supported_local_configs:
-        os.system(
-            f'GIT_DIR="{paths.repo}" '
-            f'git config --local "{config}" "value_of_{config}"')
+        os.system(f'GIT_DIR="{paths.repo}" ' f'git config --local "{config}" "value_of_{config}"')
 
     # run yadm config
     for config in supported_local_configs:
-        run = runner(yadm_y('config', config))
+        run = runner(yadm_cmd("config", config))
         assert run.success
-        assert run.err == ''
-        assert run.out.strip() == f'value_of_{config}'
+        assert run.err == ""
+        assert run.out.strip() == f"value_of_{config}"
 
 
-@pytest.mark.usefixtures('ds1_repo_copy')
-def test_config_local_write(runner, yadm_y, paths, supported_local_configs):
+@pytest.mark.usefixtures("ds1_repo_copy")
+def test_config_local_write(runner, yadm_cmd, paths, supported_local_configs):
     """Write local attribute
 
     Display no output
@@ -124,16 +123,37 @@ def test_config_local_write(runner, yadm_y, paths, supported_local_configs):
 
     # run yadm config
     for config in supported_local_configs:
-        run = runner(yadm_y('config', config, f'value_of_{config}'))
+        run = runner(yadm_cmd("config", config, f"value_of_{config}"))
         assert run.success
-        assert run.err == ''
-        assert run.out == ''
+        assert run.err == ""
+        assert run.out == ""
 
     # verify test values
     for config in supported_local_configs:
-        run = runner(
-            command=('git', 'config', config),
-            env={'GIT_DIR': paths.repo})
+        run = runner(command=("git", "config", config), env={"GIT_DIR": paths.repo})
         assert run.success
-        assert run.err == ''
-        assert run.out.strip() == f'value_of_{config}'
+        assert run.err == ""
+        assert run.out.strip() == f"value_of_{config}"
+
+
+def test_config_without_parent_directory(runner, yadm_cmd, paths):
+    """Write/read attribute to/from config file with non-existent parent dir
+
+    Update configuration file
+    Display value
+    Exit with 0
+    """
+
+    config_file = paths.root + "/folder/does/not/exist/config"
+
+    run = runner(yadm_cmd("--yadm-config", config_file, "config", TEST_KEY, TEST_VALUE))
+
+    assert run.success
+    assert run.err == ""
+    assert run.out == ""
+
+    run = runner(yadm_cmd("--yadm-config", config_file, "config", TEST_KEY))
+
+    assert run.success
+    assert run.err == ""
+    assert run.out.strip() == TEST_VALUE

test/test_encryption.py

@@ -1,42 +1,62 @@
 """Test encryption"""
 
 import os
-import pipes
+import shlex
+import time
+
 import pytest
 
-KEY_FILE = 'test/test_key'
-KEY_FINGERPRINT = 'F8BBFC746C58945442349BCEBA54FFD04C599B1A'
-KEY_NAME = 'yadm-test1'
-KEY_TRUST = 'test/ownertrust.txt'
-PASSPHRASE = 'ExamplePassword'
+KEY_FILE = "test/test_key"
+KEY_FINGERPRINT = "F8BBFC746C58945442349BCEBA54FFD04C599B1A"
+KEY_NAME = "yadm-test1"
+KEY_TRUST = "test/ownertrust.txt"
+PASSPHRASE = "ExamplePassword"
 
-pytestmark = pytest.mark.usefixtures('config_git')
+pytestmark = pytest.mark.usefixtures("config_git")
 
 
-def add_asymmetric_key():
+def add_asymmetric_key(runner, gnupg):
     """Add asymmetric key"""
-    os.system(f'gpg --import {pipes.quote(KEY_FILE)}')
-    os.system(f'gpg --import-ownertrust < {pipes.quote(KEY_TRUST)}')
+    env = os.environ.copy()
+    env["GNUPGHOME"] = gnupg.home
+    runner(
+        ["gpg", "--import", shlex.quote(KEY_FILE)],
+        env=env,
+        shell=True,
+    )
+    runner(
+        ["gpg", "--import-ownertrust", "<", shlex.quote(KEY_TRUST)],
+        env=env,
+        shell=True,
+    )
 
 
-def remove_asymmetric_key():
+def remove_asymmetric_key(runner, gnupg):
     """Remove asymmetric key"""
-    os.system(
-        f'gpg --batch --yes '
-        f'--delete-secret-keys {pipes.quote(KEY_FINGERPRINT)}')
-    os.system(f'gpg --batch --yes --delete-key {pipes.quote(KEY_FINGERPRINT)}')
+    env = os.environ.copy()
+    env["GNUPGHOME"] = gnupg.home
+    runner(
+        ["gpg", "--batch", "--yes", "--delete-secret-keys", shlex.quote(KEY_FINGERPRINT)],
+        env=env,
+        shell=True,
+    )
+    runner(
+        ["gpg", "--batch", "--yes", "--delete-key", shlex.quote(KEY_FINGERPRINT)],
+        env=env,
+        shell=True,
+    )
 
 
 @pytest.fixture
-def asymmetric_key():
+def asymmetric_key(runner, gnupg):
     """Fixture for asymmetric key, removed in teardown"""
-    add_asymmetric_key()
+    add_asymmetric_key(runner, gnupg)
     yield KEY_NAME
-    remove_asymmetric_key()
+    remove_asymmetric_key(runner, gnupg)
 
 
 @pytest.fixture
-def encrypt_targets(yadm_y, paths):
+def encrypt_targets(yadm_cmd, paths):
     """Fixture for setting up data to encrypt
 
     This fixture:
@@ -53,49 +73,49 @@ def encrypt_targets(yadm_y, paths):
     """
 
     # init empty yadm repo
-    os.system(' '.join(yadm_y('init', '-w', str(paths.work), '-f')))
+    os.system(" ".join(yadm_cmd("init", "-w", str(paths.work), "-f")))
 
     expected = []
 
     # standard files w/ dirs & spaces
-    paths.work.join('inc file1').write('inc file1')
-    expected.append('inc file1')
-    paths.encrypt.write('inc file1\n')
-    paths.work.join('inc dir').mkdir()
-    paths.work.join('inc dir/inc file2').write('inc file2')
-    expected.append('inc dir/inc file2')
-    paths.encrypt.write('inc dir/inc file2\n', mode='a')
+    paths.work.join("inc file1").write("inc file1")
+    expected.append("inc file1")
+    paths.encrypt.write("inc file1\n")
+    paths.work.join("inc dir").mkdir()
+    paths.work.join("inc dir/inc file2").write("inc file2")
+    expected.append("inc dir/inc file2")
+    paths.encrypt.write("inc dir/inc file2\n", mode="a")
 
     # standard globs w/ dirs & spaces
-    paths.work.join('globs file1').write('globs file1')
-    expected.append('globs file1')
-    paths.work.join('globs dir').mkdir()
-    paths.work.join('globs dir/globs file2').write('globs file2')
-    expected.append('globs dir/globs file2')
-    paths.encrypt.write('globs*\n', mode='a')
+    paths.work.join("globs file1").write("globs file1")
+    expected.append("globs file1")
+    paths.work.join("globs dir").mkdir()
+    paths.work.join("globs dir/globs file2").write("globs file2")
+    expected.append("globs dir/globs file2")
+    paths.encrypt.write("globs*\n", mode="a")
 
     # blank lines
-    paths.encrypt.write('\n        \n\t\n', mode='a')
+    paths.encrypt.write("\n        \n\t\n", mode="a")
 
     # comments
-    paths.work.join('commentfile1').write('commentfile1')
-    paths.encrypt.write('#commentfile1\n', mode='a')
-    paths.encrypt.write('        #commentfile1\n', mode='a')
+    paths.work.join("commentfile1").write("commentfile1")
+    paths.encrypt.write("#commentfile1\n", mode="a")
+    paths.encrypt.write("        #commentfile1\n", mode="a")
 
     # exclusions
-    paths.work.join('extest').mkdir()
-    paths.encrypt.write('extest/*\n', mode='a')  # include within extest
-    paths.work.join('extest/inglob1').write('inglob1')
-    paths.work.join('extest/exglob1').write('exglob1')
-    paths.work.join('extest/exglob2').write('exglob2')
-    paths.encrypt.write('!extest/ex*\n', mode='a')  # exclude the ex*
-    expected.append('extest/inglob1')  # should be left with only in*
+    paths.work.join("extest").mkdir()
+    paths.encrypt.write("extest/*\n", mode="a")  # include within extest
+    paths.work.join("extest/inglob1").write("inglob1")
+    paths.work.join("extest/exglob1").write("exglob1")
+    paths.work.join("extest/exglob2").write("exglob2")
+    paths.encrypt.write("!extest/ex*\n", mode="a")  # exclude the ex*
+    expected.append("extest/inglob1")  # should be left with only in*
 
     return expected
 
 
-@pytest.fixture(scope='session')
-def decrypt_targets(tmpdir_factory, runner):
+@pytest.fixture(scope="session")
+def decrypt_targets(tmpdir_factory, runner, gnupg):
     """Fixture for setting data to decrypt
 
     This fixture:
@@ -103,231 +123,232 @@ def decrypt_targets(tmpdir_factory, runner):
       * creates a list of expected decrypted files
     """
 
-    tmpdir = tmpdir_factory.mktemp('decrypt_targets')
-    symmetric = tmpdir.join('symmetric.tar.gz.gpg')
-    asymmetric = tmpdir.join('asymmetric.tar.gz.gpg')
+    tmpdir = tmpdir_factory.mktemp("decrypt_targets")
+    symmetric = tmpdir.join("symmetric.tar.gz.gpg")
+    asymmetric = tmpdir.join("asymmetric.tar.gz.gpg")
 
     expected = []
 
-    tmpdir.join('decrypt1').write('decrypt1')
-    expected.append('decrypt1')
-    tmpdir.join('decrypt2').write('decrypt2')
-    expected.append('decrypt2')
-    tmpdir.join('subdir').mkdir()
-    tmpdir.join('subdir/decrypt3').write('subdir/decrypt3')
-    expected.append('subdir/decrypt3')
+    tmpdir.join("decrypt1").write("decrypt1")
+    expected.append("decrypt1")
+    tmpdir.join("decrypt2").write("decrypt2")
+    expected.append("decrypt2")
+    tmpdir.join("subdir").mkdir()
+    tmpdir.join("subdir/decrypt3").write("subdir/decrypt3")
+    expected.append("subdir/decrypt3")
 
+    gnupg.pw(PASSPHRASE)
+    env = os.environ.copy()
+    env["GNUPGHOME"] = gnupg.home
     run = runner(
-        ['tar', 'cvf', '-'] +
-        expected +
-        ['|', 'gpg', '--batch', '--yes', '-c'] +
-        ['--passphrase', pipes.quote(PASSPHRASE)] +
-        ['--output', pipes.quote(str(symmetric))],
+        ["tar", "cvf", "-"]
+        + expected
+        + ["|", "gpg", "--batch", "--yes", "-c"]
+        + ["--output", shlex.quote(str(symmetric))],
         cwd=tmpdir,
-        shell=True)
+        env=env,
+        shell=True,
+    )
     assert run.success
 
-    add_asymmetric_key()
+    gnupg.pw("")
+    add_asymmetric_key(runner, gnupg)
     run = runner(
-        ['tar', 'cvf', '-'] +
-        expected +
-        ['|', 'gpg', '--batch', '--yes', '-e'] +
-        ['-r', pipes.quote(KEY_NAME)] +
-        ['--output', pipes.quote(str(asymmetric))],
+        ["tar", "cvf", "-"]
+        + expected
+        + ["|", "gpg", "--batch", "--yes", "-e"]
+        + ["-r", shlex.quote(KEY_NAME)]
+        + ["--output", shlex.quote(str(asymmetric))],
         cwd=tmpdir,
-        shell=True)
+        env=env,
+        shell=True,
+    )
     assert run.success
-    remove_asymmetric_key()
+    remove_asymmetric_key(runner, gnupg)
 
     return {
-        'asymmetric': asymmetric,
-        'expected': expected,
-        'symmetric': symmetric,
+        "asymmetric": asymmetric,
+        "expected": expected,
+        "symmetric": symmetric,
     }
 
 
-@pytest.mark.parametrize(
-    'mismatched_phrase', [False, True],
-    ids=['matching_phrase', 'mismatched_phrase'])
-@pytest.mark.parametrize(
-    'missing_encrypt', [False, True],
-    ids=['encrypt_exists', 'encrypt_missing'])
-@pytest.mark.parametrize(
-    'overwrite', [False, True],
-    ids=['clean', 'overwrite'])
-def test_symmetric_encrypt(
-        runner, yadm_y, paths, encrypt_targets,
-        overwrite, missing_encrypt, mismatched_phrase):
+@pytest.mark.parametrize("bad_phrase", [False, True], ids=["good_phrase", "bad_phrase"])
+@pytest.mark.parametrize("missing_encrypt", [False, True], ids=["encrypt_exists", "encrypt_missing"])
+@pytest.mark.parametrize("overwrite", [False, True], ids=["clean", "overwrite"])
+def test_symmetric_encrypt(runner, yadm_cmd, paths, encrypt_targets, gnupg, bad_phrase, overwrite, missing_encrypt):
     """Test symmetric encryption"""
 
     if missing_encrypt:
         paths.encrypt.remove()
 
-    matched_phrase = PASSPHRASE
-    if mismatched_phrase:
-        matched_phrase = 'mismatched'
+    if bad_phrase:
+        gnupg.pw("")
+    else:
+        gnupg.pw(PASSPHRASE)
 
     if overwrite:
-        paths.archive.write('existing archive')
+        paths.archive.write("existing archive")
 
-    run = runner(yadm_y('encrypt'), expect=[
-        ('passphrase:', PASSPHRASE),
-        ('passphrase:', matched_phrase),
-        ])
+    env = os.environ.copy()
+    env["GNUPGHOME"] = gnupg.home
+    run = runner(yadm_cmd("encrypt"), env=env)
 
-    if missing_encrypt or mismatched_phrase:
+    if missing_encrypt or bad_phrase:
         assert run.failure
     else:
         assert run.success
-        assert run.err == ''
+        assert run.err == ""
 
     if missing_encrypt:
-        assert 'does not exist' in run.out
-    elif mismatched_phrase:
-        assert 'invalid passphrase' in run.out
+        assert "does not exist" in run.err
+    elif bad_phrase:
+        assert "Invalid IPC" in run.err
     else:
-        assert encrypted_data_valid(runner, paths.archive, encrypt_targets)
+        assert encrypted_data_valid(runner, gnupg, paths.archive, encrypt_targets)
 
 
-@pytest.mark.parametrize(
-    'wrong_phrase', [False, True],
-    ids=['correct_phrase', 'wrong_phrase'])
-@pytest.mark.parametrize(
-    'archive_exists', [True, False],
-    ids=['archive_exists', 'archive_missing'])
-@pytest.mark.parametrize(
-    'dolist', [False, True],
-    ids=['decrypt', 'list'])
-def test_symmetric_decrypt(
-        runner, yadm_y, paths, decrypt_targets,
-        dolist, archive_exists, wrong_phrase):
+@pytest.mark.parametrize("bad_phrase", [False, True], ids=["good_phrase", "bad_phrase"])
+@pytest.mark.parametrize("archive_exists", [True, False], ids=["archive_exists", "archive_missing"])
+@pytest.mark.parametrize("dolist", [False, True], ids=["decrypt", "list"])
+def test_symmetric_decrypt(runner, yadm_cmd, paths, decrypt_targets, gnupg, dolist, archive_exists, bad_phrase):
     """Test decryption"""
 
     # init empty yadm repo
-    os.system(' '.join(yadm_y('init', '-w', str(paths.work), '-f')))
+    os.system(" ".join(yadm_cmd("init", "-w", str(paths.work), "-f")))
 
-    phrase = PASSPHRASE
-    if wrong_phrase:
-        phrase = 'wrong-phrase'
+    if bad_phrase:
+        gnupg.pw("")
+        time.sleep(1)  # allow gpg-agent cache to expire
+    else:
+        gnupg.pw(PASSPHRASE)
 
     if archive_exists:
-        decrypt_targets['symmetric'].copy(paths.archive)
+        decrypt_targets["symmetric"].copy(paths.archive)
 
     # to test overwriting
-    paths.work.join('decrypt1').write('pre-existing file')
+    paths.work.join("decrypt1").write("pre-existing file")
+
+    env = os.environ.copy()
+    env["GNUPGHOME"] = gnupg.home
 
     args = []
 
     if dolist:
-        args.append('-l')
-    run = runner(yadm_y('decrypt') + args, expect=[('passphrase:', phrase)])
+        args.append("-l")
+    run = runner(yadm_cmd("decrypt") + args, env=env)
 
-    if archive_exists and not wrong_phrase:
+    if archive_exists and not bad_phrase:
         assert run.success
-        assert run.err == ''
+        assert "encrypted with 1 passphrase" in run.err
         if dolist:
-            for filename in decrypt_targets['expected']:
-                if filename != 'decrypt1':  # this one should exist
+            for filename in decrypt_targets["expected"]:
+                if filename != "decrypt1":  # this one should exist
                     assert not paths.work.join(filename).exists()
                 assert filename in run.out
         else:
-            for filename in decrypt_targets['expected']:
+            for filename in decrypt_targets["expected"]:
                 assert paths.work.join(filename).read() == filename
     else:
         assert run.failure
 
 
-@pytest.mark.usefixtures('asymmetric_key')
-@pytest.mark.parametrize(
-    'ask', [False, True],
-    ids=['no_ask', 'ask'])
-@pytest.mark.parametrize(
-    'key_exists', [True, False],
-    ids=['key_exists', 'key_missing'])
-@pytest.mark.parametrize(
-    'overwrite', [False, True],
-    ids=['clean', 'overwrite'])
-def test_asymmetric_encrypt(
-        runner, yadm_y, paths, encrypt_targets,
-        overwrite, key_exists, ask):
+@pytest.mark.usefixtures("asymmetric_key")
+@pytest.mark.parametrize("ask", [False, True], ids=["no_ask", "ask"])
+@pytest.mark.parametrize("key_exists", [True, False], ids=["key_exists", "key_missing"])
+@pytest.mark.parametrize("overwrite", [False, True], ids=["clean", "overwrite"])
+def test_asymmetric_encrypt(runner, yadm_cmd, paths, encrypt_targets, gnupg, overwrite, key_exists, ask):
     """Test asymmetric encryption"""
 
     # specify encryption recipient
     if ask:
-        os.system(' '.join(yadm_y('config', 'yadm.gpg-recipient', 'ASK')))
-        expect = [('Enter the user ID', KEY_NAME), ('Enter the user ID', '')]
+        os.system(" ".join(yadm_cmd("config", "yadm.gpg-recipient", "ASK")))
+        expect = [("Enter the user ID", KEY_NAME), ("Enter the user ID", "")]
     else:
-        os.system(' '.join(yadm_y('config', 'yadm.gpg-recipient', KEY_NAME)))
+        os.system(" ".join(yadm_cmd("config", "yadm.gpg-recipient", KEY_NAME)))
         expect = []
 
     if overwrite:
-        paths.archive.write('existing archive')
+        paths.archive.write("existing archive")
 
     if not key_exists:
-        remove_asymmetric_key()
+        remove_asymmetric_key(runner, gnupg)
 
-    run = runner(yadm_y('encrypt'), expect=expect)
+    env = os.environ.copy()
+    env["GNUPGHOME"] = gnupg.home
+
+    run = runner(yadm_cmd("encrypt"), env=env, expect=expect)
 
     if key_exists:
         assert run.success
-        assert encrypted_data_valid(runner, paths.archive, encrypt_targets)
+        assert encrypted_data_valid(runner, gnupg, paths.archive, encrypt_targets)
     else:
         assert run.failure
-        assert 'Unable to write' in run.out
+        assert "Unable to write" in run.out if expect else run.err
 
     if ask:
-        assert 'Enter the user ID' in run.out
+        assert "Enter the user ID" in run.out
 
 
-@pytest.mark.usefixtures('asymmetric_key')
-@pytest.mark.parametrize(
-    'key_exists', [True, False],
-    ids=['key_exists', 'key_missing'])
-@pytest.mark.parametrize(
-    'dolist', [False, True],
-    ids=['decrypt', 'list'])
-def test_asymmetric_decrypt(
-        runner, yadm_y, paths, decrypt_targets,
-        dolist, key_exists):
+@pytest.mark.usefixtures("asymmetric_key")
+@pytest.mark.usefixtures("encrypt_targets")
+def test_multi_key(runner, yadm_cmd, gnupg):
+    """Test multiple recipients"""
+
+    # specify two encryption recipient
+    os.system(" ".join(yadm_cmd("config", "yadm.gpg-recipient", f'"second-key {KEY_NAME}"')))
+
+    env = os.environ.copy()
+    env["GNUPGHOME"] = gnupg.home
+
+    run = runner(yadm_cmd("encrypt"), env=env)
+
+    assert run.failure
+    assert "second-key: skipped: No public key" in run.err
+
+
+@pytest.mark.usefixtures("asymmetric_key")
+@pytest.mark.parametrize("key_exists", [True, False], ids=["key_exists", "key_missing"])
+@pytest.mark.parametrize("dolist", [False, True], ids=["decrypt", "list"])
+def test_asymmetric_decrypt(runner, yadm_cmd, paths, decrypt_targets, gnupg, dolist, key_exists):
     """Test decryption"""
 
     # init empty yadm repo
-    os.system(' '.join(yadm_y('init', '-w', str(paths.work), '-f')))
+    os.system(" ".join(yadm_cmd("init", "-w", str(paths.work), "-f")))
 
-    decrypt_targets['asymmetric'].copy(paths.archive)
+    decrypt_targets["asymmetric"].copy(paths.archive)
 
     # to test overwriting
-    paths.work.join('decrypt1').write('pre-existing file')
+    paths.work.join("decrypt1").write("pre-existing file")
 
     if not key_exists:
-        remove_asymmetric_key()
+        remove_asymmetric_key(runner, gnupg)
 
     args = []
 
     if dolist:
-        args.append('-l')
-    run = runner(yadm_y('decrypt') + args)
+        args.append("-l")
+    env = os.environ.copy()
+    env["GNUPGHOME"] = gnupg.home
+    run = runner(yadm_cmd("decrypt") + args, env=env)
 
     if key_exists:
         assert run.success
         if dolist:
-            for filename in decrypt_targets['expected']:
-                if filename != 'decrypt1':  # this one should exist
+            for filename in decrypt_targets["expected"]:
+                if filename != "decrypt1":  # this one should exist
                     assert not paths.work.join(filename).exists()
                 assert filename in run.out
         else:
-            for filename in decrypt_targets['expected']:
+            for filename in decrypt_targets["expected"]:
                 assert paths.work.join(filename).read() == filename
     else:
         assert run.failure
-        assert 'Unable to extract encrypted files' in run.out
+        assert "Unable to extract encrypted files" in run.err
 
 
-@pytest.mark.parametrize(
-    'untracked',
-    [False, 'y', 'n'],
-    ids=['tracked', 'untracked_answer_y', 'untracked_answer_n'])
-def test_offer_to_add(runner, yadm_y, paths, encrypt_targets, untracked):
+@pytest.mark.parametrize("untracked", [False, "y", "n"], ids=["tracked", "untracked_answer_y", "untracked_answer_n"])
+def test_offer_to_add(runner, yadm_cmd, paths, encrypt_targets, gnupg, untracked):
     """Test offer to add encrypted archive
 
     All the other encryption tests use an archive outside of the work tree.
@@ -335,81 +356,75 @@ def test_offer_to_add(runner, yadm_y, paths, encrypt_targets, untracked):
     should be an offer to add it to the repo if it is not tracked.
     """
 
-    worktree_archive = paths.work.join('worktree-archive.tar.gpg')
-    expect = [
-        ('passphrase:', PASSPHRASE),
-        ('passphrase:', PASSPHRASE),
-        ]
+    worktree_archive = paths.work.join("worktree-archive.tar.gpg")
+
+    expect = []
+
+    gnupg.pw(PASSPHRASE)
+    env = os.environ.copy()
+    env["GNUPGHOME"] = gnupg.home
 
     if untracked:
-        expect.append(('add it now', untracked))
+        expect.append(("add it now", untracked))
     else:
-        worktree_archive.write('exists')
-        os.system(' '.join(yadm_y('add', str(worktree_archive))))
+        worktree_archive.write("exists")
+        os.system(" ".join(yadm_cmd("add", str(worktree_archive))))
 
-    run = runner(
-        yadm_y('encrypt', '--yadm-archive', str(worktree_archive)),
-        expect=expect
-        )
+    run = runner(yadm_cmd("encrypt", "--yadm-archive", str(worktree_archive)), env=env, expect=expect)
 
     assert run.success
-    assert run.err == ''
-    assert encrypted_data_valid(runner, worktree_archive, encrypt_targets)
+    assert run.err == ""
+    assert encrypted_data_valid(runner, gnupg, worktree_archive, encrypt_targets)
 
-    run = runner(
-        yadm_y('status', '--porcelain', '-uall', str(worktree_archive)))
+    run = runner(yadm_cmd("status", "--porcelain", "-uall", str(worktree_archive)))
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
 
-    if untracked == 'y':
+    if untracked == "y":
         # should be added to the index
-        assert f'A  {worktree_archive.basename}' in run.out
-    elif untracked == 'n':
+        assert f"A  {worktree_archive.basename}" in run.out
+    elif untracked == "n":
         # should NOT be added to the index
-        assert f'?? {worktree_archive.basename}' in run.out
+        assert f"?? {worktree_archive.basename}" in run.out
     else:
         # should appear modified in the index
-        assert f'AM {worktree_archive.basename}' in run.out
+        assert f"AM {worktree_archive.basename}" in run.out
 
 
-def test_encrypt_added_to_exclude(runner, yadm_y, paths):
+@pytest.mark.usefixtures("ds1_copy")
+def test_encrypt_added_to_exclude(runner, yadm_cmd, paths, gnupg):
     """Confirm that .config/yadm/encrypt is added to exclude"""
 
-    expect = [
-        ('passphrase:', PASSPHRASE),
-        ('passphrase:', PASSPHRASE),
-        ]
+    gnupg.pw(PASSPHRASE)
+    env = os.environ.copy()
+    env["GNUPGHOME"] = gnupg.home
 
-    exclude_file = paths.repo.join('info/exclude')
-    paths.encrypt.write('test-encrypt-data\n')
-    exclude_file.write('original-data', ensure=True)
+    exclude_file = paths.repo.join("info/exclude")
+    paths.encrypt.write("test-encrypt-data\n")
+    paths.work.join("test-encrypt-data").write("")
+    exclude_file.write("original-data", ensure=True)
 
-    run = runner(
-        yadm_y('encrypt'),
-        expect=expect,
-        )
+    run = runner(yadm_cmd("encrypt"), env=env)
 
-    assert 'test-encrypt-data' in paths.repo.join('info/exclude').read()
-    assert 'original-data' in paths.repo.join('info/exclude').read()
+    assert "test-encrypt-data" in paths.repo.join("info/exclude").read()
+    assert "original-data" in paths.repo.join("info/exclude").read()
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
 
 
-def encrypted_data_valid(runner, encrypted, expected):
+def encrypted_data_valid(runner, gnupg, encrypted, expected):
     """Verify encrypted data matches expectations"""
-    run = runner([
-        'gpg',
-        '--passphrase', pipes.quote(PASSPHRASE),
-        '-d', pipes.quote(str(encrypted)),
-        '2>/dev/null',
-        '|', 'tar', 't'], shell=True, report=False)
+    gnupg.pw(PASSPHRASE)
+    env = os.environ.copy()
+    env["GNUPGHOME"] = gnupg.home
+    run = runner(
+        ["gpg", "-d", shlex.quote(str(encrypted)), "2>/dev/null", "|", "tar", "t"], env=env, shell=True, report=False
+    )
     file_count = 0
     for filename in run.out.splitlines():
-        if filename.endswith('/'):
+        if filename.endswith("/"):
             continue
         file_count += 1
-        assert filename in expected, (
-            f'Unexpected file in archive: {filename}')
-    assert file_count == len(expected), (
-        'Number of files in archive does not match expected')
+        assert filename in expected, f"Unexpected file in archive: {filename}"
+    assert file_count == len(expected), "Number of files in archive does not match expected"
     return True

test/test_enter.py

@@ -1,77 +1,117 @@
 """Test enter"""
 
 import os
+
 import pytest
 
 
 @pytest.mark.parametrize(
-    'shell, success', [
-        ('delete', True),  # if there is no shell variable, bash creates it
-        ('', False),
-        ('/usr/bin/env', True),
-        ('noexec', False),
-    ], ids=[
-        'shell-missing',
-        'shell-empty',
-        'shell-env',
-        'shell-noexec',
-    ])
-@pytest.mark.usefixtures('ds1_copy')
-def test_enter(runner, yadm_y, paths, shell, success):
+    "shell, success",
+    [
+        ("delete", True),  # if there is no shell variable, bash creates it
+        ("", False),
+        ("/usr/bin/env", True),
+        ("noexec", False),
+    ],
+    ids=[
+        "shell-missing",
+        "shell-empty",
+        "shell-env",
+        "shell-noexec",
+    ],
+)
+@pytest.mark.usefixtures("ds1_copy")
+def test_enter(runner, yadm_cmd, paths, shell, success):
     """Enter tests"""
     env = os.environ.copy()
-    if shell == 'delete':
+    if shell == "delete":
         # remove shell
-        if 'SHELL' in env:
-            del env['SHELL']
-    elif shell == 'noexec':
+        if "SHELL" in env:
+            del env["SHELL"]
+    elif shell == "noexec":
         # specify a non-executable path
-        noexec = paths.root.join('noexec')
-        noexec.write('')
+        noexec = paths.root.join("noexec")
+        noexec.write("")
         noexec.chmod(0o664)
-        env['SHELL'] = str(noexec)
+        env["SHELL"] = str(noexec)
     else:
-        env['SHELL'] = shell
+        env["SHELL"] = shell
 
-    run = runner(command=yadm_y('enter'), env=env)
+    run = runner(command=yadm_cmd("enter"), env=env)
     assert run.success == success
-    assert run.err == ''
-    prompt = f'yadm shell ({paths.repo})'
+    prompt = f"yadm shell ({paths.repo})"
     if success:
-        assert run.out.startswith('Entering yadm repo')
-        assert run.out.rstrip().endswith('Leaving yadm repo')
-    if not success:
-        assert 'does not refer to an executable' in run.out
-    if 'env' in shell:
-        assert f'GIT_DIR={paths.repo}' in run.out
-        assert f'PROMPT={prompt}' in run.out
-        assert f'PS1={prompt}' in run.out
+        assert run.out.startswith("Entering yadm repo")
+        assert run.out.rstrip().endswith("Leaving yadm repo")
+        assert run.err == ""
+    else:
+        assert "does not refer to an executable" in run.err
+    if "env" in shell:
+        assert f"GIT_DIR={paths.repo}" in run.out
+        assert f"GIT_WORK_TREE={paths.work}" in run.out
+        assert f"PROMPT={prompt}" in run.out
+        assert f"PS1={prompt}" in run.out
 
 
 @pytest.mark.parametrize(
-    'shell, opts, path', [
-        ('bash', '--norc', '\\w'),
-        ('csh', '-f', '%~'),
-        ('zsh', '-f', '%~'),
-    ], ids=[
-        'bash',
-        'csh',
-        'zsh',
-    ])
-@pytest.mark.usefixtures('ds1_copy')
-def test_enter_shell_ops(runner, yadm_y, paths, shell, opts, path):
+    "shell, opts, path",
+    [
+        ("bash", "--norc", "\\w"),
+        ("csh", "-f", "%~"),
+        ("zsh", "-f", "%~"),
+    ],
+    ids=[
+        "bash",
+        "csh",
+        "zsh",
+    ],
+)
+@pytest.mark.parametrize(
+    "cmd",
+    [False, "cmd", "cmd-bad-exit"],
+    ids=["no-cmd", "cmd", "cmd-bad-exit"],
+)
+@pytest.mark.parametrize(
+    "term",
+    ["", "dumb"],
+    ids=["term-empty", "term-dumb"],
+)
+@pytest.mark.usefixtures("ds1_copy")
+def test_enter_shell_ops(runner, yadm_cmd, paths, shell, opts, path, cmd, term):
     """Enter tests for specific shell options"""
 
+    change_exit = "\nfalse" if cmd == "cmd-bad-exit" else ""
+
     # Create custom shell to detect options passed
     custom_shell = paths.root.join(shell)
-    custom_shell.write('#!/bin/sh\necho OPTS=$*\necho PROMPT=$PROMPT')
+    custom_shell.write(f"#!/bin/sh\necho OPTS=$*\necho PROMPT=$PROMPT{change_exit}")
     custom_shell.chmod(0o775)
 
-    env = os.environ.copy()
-    env['SHELL'] = custom_shell
+    test_cmd = ["test1", "test2", "test3"]
 
-    run = runner(command=yadm_y('enter'), env=env)
-    assert run.success
-    assert run.err == ''
-    assert f'OPTS={opts}' in run.out
-    assert f'PROMPT=yadm shell ({paths.repo}) {path} >' in run.out
+    enter_cmd = ["enter"]
+    if cmd:
+        enter_cmd += test_cmd
+
+    env = os.environ.copy()
+    env["TERM"] = term
+    env["SHELL"] = custom_shell
+
+    if shell == "zsh" and term == "dumb":
+        opts += " --no-zle"
+
+    run = runner(command=yadm_cmd(*enter_cmd), env=env)
+    if cmd == "cmd-bad-exit":
+        assert run.failure
+    else:
+        assert run.success
+    assert run.err == ""
+    assert f"OPTS={opts}" in run.out
+    assert f"PROMPT=yadm shell ({paths.repo}) {path} >" in run.out
+    if cmd:
+        assert "-c " + " ".join(test_cmd) in run.out
+        assert "Entering yadm repo" not in run.out
+        assert "Leaving yadm repo" not in run.out
+    else:
+        assert "Entering yadm repo" in run.out
+        assert "Leaving yadm repo" in run.out

test/test_ext_crypt.py

@@ -0,0 +1,50 @@
+"""Test external encryption commands"""
+
+import pytest
+
+
+@pytest.mark.parametrize(
+    "crypt",
+    [False, "installed", "installed-but-failed"],
+    ids=["not-installed", "installed", "installed-but-failed"],
+)
+@pytest.mark.parametrize(
+    "cmd,var",
+    [
+        ["git_crypt", "GIT_CRYPT_PROGRAM"],
+        ["transcrypt", "TRANSCRYPT_PROGRAM"],
+    ],
+    ids=["git-crypt", "transcrypt"],
+)
+def test_ext_encryption(runner, yadm, paths, tmpdir, crypt, cmd, var):
+    """External encryption tests"""
+
+    paths.repo.ensure(dir=True)
+    bindir = tmpdir.mkdir("bin")
+    pgm = bindir.join("test-ext-crypt")
+
+    if crypt:
+        pgm.write("#!/bin/sh\necho ext-crypt ran\n")
+        pgm.chmod(0o775)
+    if crypt == "installed-but-failed":
+        pgm.write("false\n", mode="a")
+
+    script = f"""
+        YADM_TEST=1 source {yadm}
+        YADM_REPO={paths.repo}
+        {var}="{pgm}"
+        {cmd} "param1"
+    """
+
+    run = runner(command=["bash"], inp=script)
+
+    if crypt:
+        if crypt == "installed-but-failed":
+            assert run.failure
+        else:
+            assert run.success
+        assert run.out.strip() == "ext-crypt ran"
+        assert run.err == ""
+    else:
+        assert run.failure
+        assert f"command '{pgm}' cannot be located" in run.err

test/test_git.py

@@ -1,11 +1,12 @@
 """Test git"""
 
 import re
+
 import pytest
 
 
-@pytest.mark.usefixtures('ds1_copy')
-def test_git(runner, yadm_y, paths):
+@pytest.mark.usefixtures("ds1_copy")
+def test_git(runner, yadm_cmd, paths):
     """Test series of passthrough git commands
 
     Passthru unknown commands to Git
@@ -17,42 +18,42 @@ def test_git(runner, yadm_y, paths):
     """
 
     # passthru unknown commands to Git
-    run = runner(command=yadm_y('bogus'))
+    run = runner(command=yadm_cmd("bogus"))
     assert run.failure
     assert "git: 'bogus' is not a git command." in run.err
     assert "See 'git --help'" in run.err
-    assert run.out == ''
+    assert run.out == ""
 
     # git command 'add' - badfile
-    run = runner(command=yadm_y('add', '-v', 'does_not_exist'))
+    run = runner(command=yadm_cmd("add", "-v", "does_not_exist"))
     assert run.code == 128
     assert "pathspec 'does_not_exist' did not match any files" in run.err
-    assert run.out == ''
+    assert run.out == ""
 
     # git command 'add'
-    newfile = paths.work.join('test_git')
-    newfile.write('test_git')
-    run = runner(command=yadm_y('add', '-v', str(newfile)))
+    newfile = paths.work.join("test_git")
+    newfile.write("test_git")
+    run = runner(command=yadm_cmd("add", "-v", str(newfile)))
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
     assert "add 'test_git'" in run.out
 
     # git command 'status'
-    run = runner(command=yadm_y('status'))
+    run = runner(command=yadm_cmd("status"))
     assert run.success
-    assert run.err == ''
-    assert re.search(r'new file:\s+test_git', run.out)
+    assert run.err == ""
+    assert re.search(r"new file:\s+test_git", run.out)
 
     # git command 'commit'
-    run = runner(command=yadm_y('commit', '-m', 'Add test_git'))
+    run = runner(command=yadm_cmd("commit", "-m", "Add test_git"))
     assert run.success
-    assert run.err == ''
-    assert '1 file changed' in run.out
-    assert '1 insertion' in run.out
-    assert re.search(r'create mode .+ test_git', run.out)
+    assert run.err == ""
+    assert "1 file changed" in run.out
+    assert "1 insertion" in run.out
+    assert re.search(r"create mode .+ test_git", run.out)
 
     # git command 'log'
-    run = runner(command=yadm_y('log', '--oneline'))
+    run = runner(command=yadm_cmd("log", "--oneline"))
     assert run.success
-    assert run.err == ''
-    assert 'Add test_git' in run.out
+    assert run.err == ""
+    assert "Add test_git" in run.out

test/test_help.py

@@ -1,17 +1,19 @@
 """Test help"""
+import pytest
 
 
-def test_missing_command(runner, yadm_y):
+def test_missing_command(runner, yadm_cmd):
     """Run without any command"""
-    run = runner(command=yadm_y())
+    run = runner(command=yadm_cmd())
     assert run.failure
-    assert run.err == ''
-    assert run.out.startswith('Usage: yadm')
+    assert run.err == ""
+    assert run.out.startswith("Usage: yadm")
 
 
-def test_help_command(runner, yadm_y):
+@pytest.mark.parametrize("cmd", ["--help", "help"])
+def test_help_command(runner, yadm_cmd, cmd):
     """Run with help command"""
-    run = runner(command=yadm_y('help'))
+    run = runner(command=yadm_cmd(cmd))
     assert run.failure
-    assert run.err == ''
-    assert run.out.startswith('Usage: yadm')
+    assert run.err == ""
+    assert run.out.startswith("Usage: yadm")

test/test_hooks.py

@@ -4,7 +4,8 @@ import pytest
 
 
 @pytest.mark.parametrize(
-    'pre, pre_code, post, post_code', [
+    "pre, pre_code, post, post_code",
+    [
         (False, 0, False, 0),
         (True, 0, False, 0),
         (True, 5, False, 0),
@@ -12,79 +13,154 @@ import pytest
         (False, 0, True, 5),
         (True, 0, True, 0),
         (True, 5, True, 5),
-    ], ids=[
-        'no-hooks',
-        'pre-success',
-        'pre-fail',
-        'post-success',
-        'post-fail',
-        'pre-post-success',
-        'pre-post-fail',
-    ])
-def test_hooks(
-        runner, yadm_y, paths,
-        pre, pre_code, post, post_code):
+    ],
+    ids=[
+        "no-hooks",
+        "pre-success",
+        "pre-fail",
+        "post-success",
+        "post-fail",
+        "pre-post-success",
+        "pre-post-fail",
+    ],
+)
+@pytest.mark.parametrize("cmd", ["--version", "version"])
+def test_hooks(runner, yadm_cmd, paths, cmd, pre, pre_code, post, post_code):
     """Test pre/post hook"""
 
     # generate hooks
     if pre:
-        create_hook(paths, 'pre_version', pre_code)
+        create_hook(paths, "pre_version", pre_code)
     if post:
-        create_hook(paths, 'post_version', post_code)
+        create_hook(paths, "post_version", post_code)
 
     # run yadm
-    run = runner(yadm_y('version'))
+    run = runner(yadm_cmd(cmd))
     # when a pre hook fails, yadm should exit with the hook's code
     assert run.code == pre_code
-    assert run.err == ''
+    assert run.err == ""
 
     if pre:
-        assert 'HOOK:pre_version' in run.out
+        assert "HOOK:pre_version" in run.out
     # if pre hook is missing or successful, yadm itself should exit 0
     if run.success:
         if post:
-            assert 'HOOK:post_version' in run.out
+            assert "HOOK:post_version" in run.out
     else:
         # when a pre hook fails, yadm should not run the command
-        assert 'version will not be run' in run.out
+        assert "version will not be run" in run.out
         # when a pre hook fails, yadm should not run the post hook
-        assert 'HOOK:post_version' not in run.out
+        assert "HOOK:post_version" not in run.out
 
 
 # repo fixture is needed to test the population of YADM_HOOK_WORK
-@pytest.mark.usefixtures('ds1_repo_copy')
-def test_hook_env(runner, yadm_y, paths):
+@pytest.mark.usefixtures("ds1_repo_copy")
+def test_hook_env(runner, yadm_cmd, paths):
     """Test hook environment"""
 
     # test will be done with a non existent "git" passthru command
     # which should exit with a failing code
-    cmd = 'passthrucmd'
+    cmd = "passthrucmd"
 
     # write the hook
-    hook = paths.hooks.join(f'post_{cmd}')
-    hook.write('#!/bin/sh\nenv\n')
+    hook = paths.hooks.join(f"post_{cmd}")
+    hook.write("#!/bin/bash\nenv\ndeclare\n")
     hook.chmod(0o755)
 
-    run = runner(yadm_y(cmd, 'extra_args'))
+    run = runner(yadm_cmd(cmd, "extra_args"))
 
     # expect passthru to fail
     assert run.failure
     assert f"'{cmd}' is not a git command" in run.err
 
     # verify hook environment
-    assert 'YADM_HOOK_EXIT=1\n' in run.out
-    assert f'YADM_HOOK_COMMAND={cmd}\n' in run.out
-    assert f'YADM_HOOK_FULL_COMMAND={cmd} extra_args\n' in run.out
-    assert f'YADM_HOOK_REPO={paths.repo}\n' in run.out
-    assert f'YADM_HOOK_WORK={paths.work}\n' in run.out
+    assert "YADM_HOOK_EXIT=1\n" in run.out
+    assert f"YADM_HOOK_COMMAND={cmd}\n" in run.out
+    assert f"YADM_HOOK_DIR={paths.yadm}\n" in run.out
+    assert f"YADM_HOOK_FULL_COMMAND={cmd} extra_args\n" in run.out
+    assert f"YADM_HOOK_REPO={paths.repo}\n" in run.out
+    assert f"YADM_HOOK_WORK={paths.work}\n" in run.out
+    assert "YADM_ENCRYPT_INCLUDE_FILES=\n" in run.out
+
+    # verify the hook environment contains certain exported functions
+    for func in [
+        "builtin_dirname",
+        "relative_path",
+        "unix_path",
+        "mixed_path",
+    ]:
+        assert f"BASH_FUNC_{func}" in run.out
+
+    # verify the hook environment contains the list of encrypted files
+    script = f"""
+        YADM_TEST=1 source {paths.pgm}
+        YADM_HOOKS="{paths.hooks}"
+        HOOK_COMMAND="{cmd}"
+        ENCRYPT_INCLUDE_FILES=(a b c)
+        invoke_hook "post"
+    """
+    run = runner(command=["bash"], inp=script)
+    assert run.success
+    assert run.err == ""
+    assert "YADM_ENCRYPT_INCLUDE_FILES=a\nb\nc\n" in run.out
+
+
+def test_escaped(runner, yadm_cmd, paths):
+    """Test escaped values in YADM_HOOK_FULL_COMMAND"""
+
+    # test will be done with a non existent "git" passthru command
+    # which should exit with a failing code
+    cmd = "passthrucmd"
+
+    # write the hook
+    hook = paths.hooks.join(f"post_{cmd}")
+    hook.write("#!/bin/bash\nenv\n")
+    hook.chmod(0o755)
+
+    run = runner(yadm_cmd(cmd, "a b", "c\td", "e\\f"))
+
+    # expect passthru to fail
+    assert run.failure
+
+    # verify escaped values
+    assert f"YADM_HOOK_FULL_COMMAND={cmd} a\\ b c\\\td e\\\\f\n" in run.out
+
+
+@pytest.mark.parametrize("condition", ["exec", "no-exec", "mingw"])
+def test_executable(runner, paths, condition):
+    """Verify hook must be exectuable"""
+    cmd = "version"
+    hook = paths.hooks.join(f"pre_{cmd}")
+    hook.write("#!/bin/sh\necho HOOK\n")
+    hook.chmod(0o644)
+    if condition == "exec":
+        hook.chmod(0o755)
+
+    mingw = 'OPERATING_SYSTEM="MINGWx"' if condition == "mingw" else ""
+    script = f"""
+        YADM_TEST=1 source {paths.pgm}
+        YADM_HOOKS="{paths.hooks}"
+        HOOK_COMMAND="{cmd}"
+        {mingw}
+        invoke_hook "pre"
+    """
+    run = runner(command=["bash"], inp=script)
+
+    if condition != "mingw":
+        assert run.success
+        assert run.err == ""
+    else:
+        assert run.failure
+        assert "Permission denied" in run.err
+
+    if condition == "exec":
+        assert "HOOK" in run.out
+    elif condition == "no-exec":
+        assert "HOOK" not in run.out
 
 
 def create_hook(paths, name, code):
     """Create hook"""
     hook = paths.hooks.join(name)
-    hook.write(
-        '#!/bin/sh\n'
-        f'echo HOOK:{name}\n'
-        f'exit {code}\n'
-    )
+    hook.write("#!/bin/sh\n" f"echo HOOK:{name}\n" f"exit {code}\n")
     hook.chmod(0o755)

test/test_init.py

@@ -4,22 +4,24 @@ import pytest
 
 
 @pytest.mark.parametrize(
-    'alt_work, repo_present, force', [
+    "alt_work, repo_present, force",
+    [
         (False, False, False),
         (True, False, False),
         (False, True, False),
         (False, True, True),
         (True, True, True),
-    ], ids=[
-        'simple',
-        '-w',
-        'existing repo',
-        '-f',
-        '-w & -f',
-    ])
-@pytest.mark.usefixtures('ds1_work_copy')
-def test_init(
-        runner, yadm_y, paths, repo_config, alt_work, repo_present, force):
+    ],
+    ids=[
+        "simple",
+        "-w",
+        "existing repo",
+        "-f",
+        "-w & -f",
+    ],
+)
+@pytest.mark.usefixtures("ds1_work_copy")
+def test_init(runner, yadm_cmd, paths, repo_config, alt_work, repo_present, force):
     """Test init
 
     Repos should have attribs:
@@ -31,48 +33,54 @@ def test_init(
     """
 
     # these tests will assume this for $HOME
-    home = str(paths.root.mkdir('HOME'))
+    home = str(paths.root.mkdir("HOME"))
 
     # ds1_work_copy comes WITH an empty repo dir present.
-    old_repo = paths.repo.join('old_repo')
+    old_repo = paths.repo.join("old_repo")
     if repo_present:
         # Let's put some data in it, so we can confirm that data is gone when
         # forced to be overwritten.
-        old_repo.write('old repo data')
+        old_repo.write("old repo data")
         assert old_repo.isfile()
     else:
         paths.repo.remove()
 
     # command args
-    args = ['init']
+    args = ["init"]
+    cwd = None
     if alt_work:
-        args.extend(['-w', paths.work])
+        if force:
+            cwd = paths.work.dirname
+            args.extend(["-w", paths.work.basename])
+        else:
+            args.extend(["-w", paths.work])
     if force:
-        args.append('-f')
+        args.append("-f")
 
     # run init
-    run = runner(yadm_y(*args), env={'HOME': home})
-    assert run.err == ''
+    runner(["git", "config", "--global", "init.defaultBranch", "master"], env={"HOME": home}, cwd=cwd)
+    run = runner(yadm_cmd(*args), env={"HOME": home}, cwd=cwd)
 
     if repo_present and not force:
         assert run.failure
-        assert 'repo already exists' in run.out
-        assert old_repo.isfile(), 'Missing original repo'
+        assert "repo already exists" in run.err
+        assert old_repo.isfile(), "Missing original repo"
     else:
         assert run.success
-        assert 'Initialized empty shared Git repository' in run.out
+        assert "Initialized empty shared Git repository" in run.out
 
         if repo_present:
-            assert not old_repo.isfile(), 'Original repo still exists'
+            assert not old_repo.isfile(), "Original repo still exists"
+        else:
+            assert run.err == ""
 
         if alt_work:
-            assert repo_config('core.worktree') == paths.work
+            assert repo_config("core.worktree") == paths.work
         else:
-            assert repo_config('core.worktree') == home
+            assert repo_config("core.worktree") == home
 
         # uniform repo assertions
-        assert oct(paths.repo.stat().mode).endswith('00'), (
-            'Repo is not secure')
-        assert repo_config('core.bare') == 'false'
-        assert repo_config('status.showUntrackedFiles') == 'no'
-        assert repo_config('yadm.managed') == 'true'
+        assert oct(paths.repo.stat().mode).endswith("00"), "Repo is not secure"
+        assert repo_config("core.bare") == "false"
+        assert repo_config("status.showUntrackedFiles") == "no"
+        assert repo_config("yadm.managed") == "true"

test/test_introspect.py

@@ -4,38 +4,38 @@ import pytest
 
 
 @pytest.mark.parametrize(
-    'name', [
-        '',
-        'invalid',
-        'commands',
-        'configs',
-        'repo',
-        'switches',
-    ])
-def test_introspect_category(
-        runner, yadm_y, paths, name,
-        supported_commands, supported_configs, supported_switches):
+    "name",
+    [
+        "",
+        "invalid",
+        "commands",
+        "configs",
+        "repo",
+        "switches",
+    ],
+)
+def test_introspect_category(runner, yadm_cmd, paths, name, supported_commands, supported_configs, supported_switches):
     """Validate introspection category"""
     if name:
-        run = runner(command=yadm_y('introspect', name))
+        run = runner(command=yadm_cmd("introspect", name))
     else:
-        run = runner(command=yadm_y('introspect'))
+        run = runner(command=yadm_cmd("introspect"))
 
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
 
     expected = []
-    if name == 'commands':
+    if name == "commands":
         expected = supported_commands
-    elif name == 'configs':
+    elif name == "configs":
         expected = supported_configs
-    elif name == 'switches':
+    elif name == "switches":
         expected = supported_switches
 
     # assert values
-    if name in ('', 'invalid'):
-        assert run.out == ''
-    if name == 'repo':
+    if name in ("", "invalid"):
+        assert run.out == ""
+    if name == "repo":
         assert run.out.rstrip() == paths.repo
 
     # make sure every expected value is present

test/test_list.py

@@ -1,31 +1,34 @@
 """Test list"""
 
 import os
+
 import pytest
 
 
 @pytest.mark.parametrize(
-    'location', [
-        'work',
-        'outside',
-        'subdir',
-    ])
-@pytest.mark.usefixtures('ds1_copy')
-def test_list(runner, yadm_y, paths, ds1, location):
+    "location",
+    [
+        "work",
+        "outside",
+        "subdir",
+    ],
+)
+@pytest.mark.usefixtures("ds1_copy")
+def test_list(runner, yadm_cmd, paths, ds1, location):
     """List tests"""
-    if location == 'work':
+    if location == "work":
         run_dir = paths.work
-    elif location == 'outside':
-        run_dir = paths.work.join('..')
-    elif location == 'subdir':
+    elif location == "outside":
+        run_dir = paths.work.join("..")
+    elif location == "subdir":
         # first directory with tracked data
         run_dir = paths.work.join(ds1.tracked_dirs[0])
     with run_dir.as_cwd():
         # test with '-a'
         # should get all tracked files, relative to the work path
-        run = runner(command=yadm_y('list', '-a'))
+        run = runner(command=yadm_cmd("list", "-a"))
         assert run.success
-        assert run.err == ''
+        assert run.err == ""
         returned_files = set(run.out.splitlines())
         expected_files = {e.path for e in ds1 if e.tracked}
         assert returned_files == expected_files
@@ -33,16 +36,14 @@ def test_list(runner, yadm_y, paths, ds1, location):
         # should get all tracked files, relative to the work path unless in a
         # subdir, then those should be a limited set of files, relative to the
         # subdir
-        run = runner(command=yadm_y('list'))
+        run = runner(command=yadm_cmd("list"))
         assert run.success
-        assert run.err == ''
+        assert run.err == ""
         returned_files = set(run.out.splitlines())
-        if location == 'subdir':
+        if location == "subdir":
             basepath = os.path.basename(os.getcwd())
             # only expect files within the subdir
             # names should be relative to subdir
-            expected_files = {
-                e.path[len(basepath)+1:]
-                for e in ds1 if e.tracked and e.path.startswith(basepath)
-            }
+            index = len(basepath) + 1
+            expected_files = {e.path[index:] for e in ds1 if e.tracked and e.path.startswith(basepath)}
         assert returned_files == expected_files

test/test_perms.py

@@ -1,20 +1,21 @@
 """Test perms"""
 
 import os
+
 import pytest
 
 
-@pytest.mark.parametrize('autoperms', ['notest', 'unset', 'true', 'false'])
-@pytest.mark.usefixtures('ds1_copy')
-def test_perms(runner, yadm_y, paths, ds1, autoperms):
+@pytest.mark.parametrize("autoperms", ["notest", "unset", "true", "false"])
+@pytest.mark.usefixtures("ds1_copy")
+def test_perms(runner, yadm_cmd, paths, ds1, autoperms):
     """Test perms"""
     # set the value of auto-perms
-    if autoperms != 'notest':
-        if autoperms != 'unset':
-            os.system(' '.join(yadm_y('config', 'yadm.auto-perms', autoperms)))
+    if autoperms != "notest":
+        if autoperms != "unset":
+            os.system(" ".join(yadm_cmd("config", "yadm.auto-perms", autoperms)))
 
     # privatepaths will hold all paths that should become secured
-    privatepaths = [paths.work.join('.ssh'), paths.work.join('.gnupg')]
+    privatepaths = [paths.work.join(".ssh"), paths.work.join(".gnupg")]
     privatepaths += [paths.work.join(private.path) for private in ds1.private]
 
     # create an archive file
@@ -22,82 +23,71 @@ def test_perms(runner, yadm_y, paths, ds1, autoperms):
     privatepaths.append(paths.archive)
 
     # create encrypted file test data
-    efile1 = paths.work.join('efile1')
-    efile1.write('efile1')
-    efile2 = paths.work.join('efile2')
-    efile2.write('efile2')
-    paths.encrypt.write('efile1\nefile2\n!efile1\n')
+    efile1 = paths.work.join("efile1")
+    efile1.write("efile1")
+    efile2 = paths.work.join("efile2")
+    efile2.write("efile2")
+    paths.encrypt.write("efile1\nefile2\n!efile1\n")
     insecurepaths = [efile1]
     privatepaths.append(efile2)
 
     # assert these paths begin unsecured
     for private in privatepaths + insecurepaths:
-        assert not oct(private.stat().mode).endswith('00'), (
-            'Path started secured')
+        assert not oct(private.stat().mode).endswith("00"), "Path started secured"
 
-    cmd = 'perms'
-    if autoperms != 'notest':
-        cmd = 'status'
-    run = runner(yadm_y(cmd))
+    cmd = "perms"
+    if autoperms != "notest":
+        cmd = "status"
+    run = runner(yadm_cmd(cmd), env={"HOME": paths.work})
     assert run.success
-    assert run.err == ''
-    if cmd == 'perms':
-        assert run.out == ''
+    assert run.err == ""
+    if cmd == "perms":
+        assert run.out == ""
 
     # these paths should be secured if processing perms
     for private in privatepaths:
-        if autoperms == 'false':
-            assert not oct(private.stat().mode).endswith('00'), (
-                'Path should not be secured')
+        if autoperms == "false":
+            assert not oct(private.stat().mode).endswith("00"), "Path should not be secured"
         else:
-            assert oct(private.stat().mode).endswith('00'), (
-                'Path has not been secured')
+            assert oct(private.stat().mode).endswith("00"), "Path has not been secured"
 
     # these paths should never be secured
     for private in insecurepaths:
-        assert not oct(private.stat().mode).endswith('00'), (
-            'Path should not be secured')
+        assert not oct(private.stat().mode).endswith("00"), "Path should not be secured"
 
 
-@pytest.mark.parametrize('sshperms', [None, 'true', 'false'])
-@pytest.mark.parametrize('gpgperms', [None, 'true', 'false'])
-@pytest.mark.usefixtures('ds1_copy')
-def test_perms_control(runner, yadm_y, paths, ds1, sshperms, gpgperms):
+@pytest.mark.parametrize("sshperms", [None, "true", "false"])
+@pytest.mark.parametrize("gpgperms", [None, "true", "false"])
+@pytest.mark.usefixtures("ds1_copy")
+def test_perms_control(runner, yadm_cmd, paths, ds1, sshperms, gpgperms):
     """Test fine control of perms"""
     # set the value of ssh-perms
     if sshperms:
-        os.system(' '.join(yadm_y('config', 'yadm.ssh-perms', sshperms)))
+        os.system(" ".join(yadm_cmd("config", "yadm.ssh-perms", sshperms)))
 
     # set the value of gpg-perms
     if gpgperms:
-        os.system(' '.join(yadm_y('config', 'yadm.gpg-perms', gpgperms)))
+        os.system(" ".join(yadm_cmd("config", "yadm.gpg-perms", gpgperms)))
 
     # privatepaths will hold all paths that should become secured
-    privatepaths = [paths.work.join('.ssh'), paths.work.join('.gnupg')]
+    privatepaths = [paths.work.join(".ssh"), paths.work.join(".gnupg")]
     privatepaths += [paths.work.join(private.path) for private in ds1.private]
 
     # assert these paths begin unsecured
     for private in privatepaths:
-        assert not oct(private.stat().mode).endswith('00'), (
-            'Path started secured')
+        assert not oct(private.stat().mode).endswith("00"), "Path started secured"
 
-    run = runner(yadm_y('perms'))
+    run = runner(yadm_cmd("perms"), env={"HOME": paths.work})
     assert run.success
-    assert run.err == ''
-    assert run.out == ''
+    assert run.err == ""
+    assert run.out == ""
 
     # these paths should be secured if processing perms
     for private in privatepaths:
-        if (
-                (sshperms == 'false' and 'ssh' in str(private))
-                or
-                (gpgperms == 'false' and 'gnupg' in str(private))
-        ):
-            assert not oct(private.stat().mode).endswith('00'), (
-                'Path should not be secured')
+        if (sshperms == "false" and "ssh" in str(private)) or (gpgperms == "false" and "gnupg" in str(private)):
+            assert not oct(private.stat().mode).endswith("00"), "Path should not be secured"
         else:
-            assert oct(private.stat().mode).endswith('00'), (
-                'Path has not been secured')
+            assert oct(private.stat().mode).endswith("00"), "Path has not been secured"
 
     # verify permissions aren't changed for the worktree
-    assert oct(paths.work.stat().mode).endswith('0755')
+    assert oct(paths.work.stat().mode).endswith("0755")

test/test_syntax.py

@@ -1,52 +1,83 @@
 """Syntax checks"""
 
 import os
+
 import pytest
 
 
 def test_yadm_syntax(runner, yadm):
     """Is syntactically valid"""
-    run = runner(command=['bash', '-n', yadm])
+    run = runner(command=["bash", "-n", yadm])
     assert run.success
 
 
-def test_shellcheck(runner, yadm, shellcheck_version):
+def test_shellcheck(pytestconfig, runner, yadm, shellcheck_version):
     """Passes shellcheck"""
-    run = runner(command=['shellcheck', '-V'], report=False)
-    if f'version: {shellcheck_version}' not in run.out:
-        pytest.skip('Unsupported shellcheck version')
-    run = runner(command=['shellcheck', '-s', 'bash', yadm])
+    if not pytestconfig.getoption("--force-linters"):
+        run = runner(command=["shellcheck", "-V"], report=False)
+        if f"version: {shellcheck_version}" not in run.out:
+            pytest.skip("Unsupported shellcheck version")
+    run = runner(command=["shellcheck", "-s", "bash", yadm])
     assert run.success
 
 
-def test_pylint(runner, pylint_version):
+def test_pylint(pytestconfig, runner, pylint_version):
     """Passes pylint"""
-    run = runner(command=['pylint', '--version'], report=False)
-    if f'pylint {pylint_version}' not in run.out:
-        pytest.skip('Unsupported pylint version')
-    pyfiles = list()
-    for tfile in os.listdir('test'):
-        if tfile.endswith('.py'):
-            pyfiles.append(f'test/{tfile}')
-    run = runner(command=['pylint'] + pyfiles)
+    if not pytestconfig.getoption("--force-linters"):
+        run = runner(command=["pylint", "--version"], report=False)
+        if f"pylint {pylint_version}" not in run.out:
+            pytest.skip("Unsupported pylint version")
+    pyfiles = []
+    for tfile in os.listdir("test"):
+        if tfile.endswith(".py"):
+            pyfiles.append(f"test/{tfile}")
+    run = runner(command=["pylint"] + pyfiles)
     assert run.success
 
 
-def test_flake8(runner, flake8_version):
+def test_isort(pytestconfig, runner, isort_version):
+    """Passes isort"""
+    if not pytestconfig.getoption("--force-linters"):
+        run = runner(command=["isort", "--version"], report=False)
+        if isort_version not in run.out:
+            pytest.skip("Unsupported isort version")
+    run = runner(command=["isort", "-c", "test"])
+    assert run.success
+
+
+def test_flake8(pytestconfig, runner, flake8_version):
     """Passes flake8"""
-    run = runner(command=['flake8', '--version'], report=False)
-    if not run.out.startswith(flake8_version):
-        pytest.skip('Unsupported flake8 version')
-    run = runner(command=['flake8', 'test'])
+    if not pytestconfig.getoption("--force-linters"):
+        run = runner(command=["flake8", "--version"], report=False)
+        if not run.out.startswith(flake8_version):
+            pytest.skip("Unsupported flake8 version")
+    run = runner(command=["flake8", "test"])
     assert run.success
 
 
-def test_yamllint(runner, yamllint_version):
+def test_black(pytestconfig, runner, black_version):
+    """Passes black"""
+    if not pytestconfig.getoption("--force-linters"):
+        run = runner(command=["black", "--version"], report=False)
+        if black_version not in run.out:
+            pytest.skip("Unsupported black version")
+    run = runner(command=["black", "--check", "test"])
+    assert run.success
+
+
+def test_yamllint(pytestconfig, runner, yamllint_version):
     """Passes yamllint"""
-    run = runner(command=['yamllint', '--version'], report=False)
-    if not run.out.strip().endswith(yamllint_version):
-        pytest.skip('Unsupported yamllint version')
-    run = runner(
-        command=['yamllint', '-s', '$(find . -name \\*.yml)'],
-        shell=True)
+    if not pytestconfig.getoption("--force-linters"):
+        run = runner(command=["yamllint", "--version"], report=False)
+        if not run.out.strip().endswith(yamllint_version):
+            pytest.skip("Unsupported yamllint version")
+    run = runner(command=["yamllint", "-s", "$(find . -name \\*.yml)"], shell=True)
     assert run.success
+
+
+def test_man(runner):
+    """Check for warnings from man"""
+    run = runner(command=["man.REAL", "--warnings", "./yadm.1"])
+    assert run.success
+    assert run.err == ""
+    assert "yadm - Yet Another Dotfiles Manager" in run.out

test/test_unit_bootstrap_available.py

@@ -8,14 +8,14 @@ def test_bootstrap_missing(runner, paths):
 
 def test_bootstrap_no_exec(runner, paths):
     """Test result of bootstrap_available, when bootstrap not executable"""
-    paths.bootstrap.write('')
+    paths.bootstrap.write("")
     paths.bootstrap.chmod(0o644)
     run_test(runner, paths, False)
 
 
 def test_bootstrap_exec(runner, paths):
     """Test result of bootstrap_available, when bootstrap executable"""
-    paths.bootstrap.write('')
+    paths.bootstrap.write("")
     paths.bootstrap.chmod(0o775)
     run_test(runner, paths, True)
 
@@ -27,7 +27,7 @@ def run_test(runner, paths, success):
         YADM_BOOTSTRAP='{paths.bootstrap}'
         bootstrap_available
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success == success
-    assert run.err == ''
-    assert run.out == ''
+    assert run.err == ""
+    assert run.out == ""

test/test_unit_choose_template_cmd.py

@@ -2,20 +2,20 @@
 import pytest
 
 
-@pytest.mark.parametrize('label', ['', 'default', 'other'])
-@pytest.mark.parametrize('awk', [True, False], ids=['awk', 'no-awk'])
+@pytest.mark.parametrize("label", ["", "default", "other"])
+@pytest.mark.parametrize("awk", [True, False], ids=["awk", "no-awk"])
 def test_kind_default(runner, yadm, awk, label):
     """Test kind: default"""
 
-    expected = 'template_default'
-    awk_avail = 'true'
+    expected = "template_default"
+    awk_avail = "true"
 
     if not awk:
-        awk_avail = 'false'
-        expected = ''
+        awk_avail = "false"
+        expected = ""
 
-    if label == 'other':
-        expected = ''
+    if label == "other":
+        expected = ""
 
     script = f"""
         YADM_TEST=1 source {yadm}
@@ -23,30 +23,30 @@ def test_kind_default(runner, yadm, awk, label):
         template="$(choose_template_cmd "{label}")"
         echo "TEMPLATE:$template"
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
-    assert f'TEMPLATE:{expected}\n' in run.out
+    assert run.err == ""
+    assert f"TEMPLATE:{expected}\n" in run.out
 
 
-@pytest.mark.parametrize('label', ['envtpl', 'j2cli', 'j2', 'other'])
-@pytest.mark.parametrize('envtpl', [True, False], ids=['envtpl', 'no-envtpl'])
-@pytest.mark.parametrize('j2cli', [True, False], ids=['j2cli', 'no-j2cli'])
+@pytest.mark.parametrize("label", ["envtpl", "j2cli", "j2", "other"])
+@pytest.mark.parametrize("envtpl", [True, False], ids=["envtpl", "no-envtpl"])
+@pytest.mark.parametrize("j2cli", [True, False], ids=["j2cli", "no-j2cli"])
 def test_kind_j2cli_envtpl(runner, yadm, envtpl, j2cli, label):
     """Test kind: j2 (both j2cli & envtpl)
 
     j2cli is preferred over envtpl if available.
     """
 
-    envtpl_avail = 'true' if envtpl else 'false'
-    j2cli_avail = 'true' if j2cli else 'false'
+    envtpl_avail = "true" if envtpl else "false"
+    j2cli_avail = "true" if j2cli else "false"
 
-    if label in ('j2cli', 'j2') and j2cli:
-        expected = 'template_j2cli'
-    elif label in ('envtpl', 'j2') and envtpl:
-        expected = 'template_envtpl'
+    if label in ("j2cli", "j2") and j2cli:
+        expected = "template_j2cli"
+    elif label in ("envtpl", "j2") and envtpl:
+        expected = "template_envtpl"
     else:
-        expected = ''
+        expected = ""
 
     script = f"""
         YADM_TEST=1 source {yadm}
@@ -55,7 +55,7 @@ def test_kind_j2cli_envtpl(runner, yadm, envtpl, j2cli, label):
         template="$(choose_template_cmd "{label}")"
         echo "TEMPLATE:$template"
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
-    assert f'TEMPLATE:{expected}\n' in run.out
+    assert run.err == ""
+    assert f"TEMPLATE:{expected}\n" in run.out

test/test_unit_configure_paths.py

@@ -2,80 +2,97 @@
 
 import pytest
 
-ARCHIVE = 'files.gpg'
-BOOTSTRAP = 'bootstrap'
-CONFIG = 'config'
-ENCRYPT = 'encrypt'
-HOME = '/testhome'
-REPO = 'repo.git'
-YDIR = '.config/yadm'
+ARCHIVE = "archive"
+BOOTSTRAP = "bootstrap"
+CONFIG = "config"
+ENCRYPT = "encrypt"
+HOME = "/testhome"
+REPO = "repo.git"
+YDIR = ".config/yadm"
+YDATA = ".local/share/yadm"
 
 
 @pytest.mark.parametrize(
-    'override, expect', [
+    "override, expect",
+    [
         (None, {}),
-        ('-Y', {}),
-        ('--yadm-repo', {'repo': 'YADM_REPO', 'git': 'GIT_DIR'}),
-        ('--yadm-config', {'config': 'YADM_CONFIG'}),
-        ('--yadm-encrypt', {'encrypt': 'YADM_ENCRYPT'}),
-        ('--yadm-archive', {'archive': 'YADM_ARCHIVE'}),
-        ('--yadm-bootstrap', {'bootstrap': 'YADM_BOOTSTRAP'}),
-    ], ids=[
-        'default',
-        'override yadm dir',
-        'override repo',
-        'override config',
-        'override encrypt',
-        'override archive',
-        'override bootstrap',
-    ])
-def test_config(runner, paths, override, expect):
+        ("-Y", {"yadm": "YADM_DIR"}),
+        ("--yadm-data", {"data": "YADM_DATA"}),
+        ("--yadm-repo", {"repo": "YADM_REPO", "git": "GIT_DIR"}),
+        ("--yadm-config", {"config": "YADM_CONFIG"}),
+        ("--yadm-encrypt", {"encrypt": "YADM_ENCRYPT"}),
+        ("--yadm-archive", {"archive": "YADM_ARCHIVE"}),
+        ("--yadm-bootstrap", {"bootstrap": "YADM_BOOTSTRAP"}),
+    ],
+    ids=[
+        "default",
+        "override yadm dir",
+        "override yadm data",
+        "override repo",
+        "override config",
+        "override encrypt",
+        "override archive",
+        "override bootstrap",
+    ],
+)
+@pytest.mark.parametrize(
+    "path",
+    [".", "./override", "override", ".override", "/override"],
+    ids=["cwd", "./relative", "relative", "hidden relative", "absolute"],
+)
+def test_config(runner, paths, override, expect, path):
     """Test configure_paths"""
-    opath = 'override'
-    matches = match_map()
-    args = []
-    if override == '-Y':
-        matches = match_map('/' + opath)
+    if path.startswith("/"):
+        expected_path = path
+    else:
+        expected_path = str(paths.root.join(path))
 
-    if override:
-        args = [override, '/' + opath]
-        for ekey in expect.keys():
-            matches[ekey] = f'{expect[ekey]}="/{opath}"'
-        run_test(
-            runner, paths,
-            [override, opath],
-            ['must specify a fully qualified'], 1)
+    args = [override, path] if override else []
 
-    run_test(runner, paths, args, matches.values(), 0)
+    if override == "-Y":
+        matches = match_map(expected_path)
+    elif override == "--yadm-data":
+        matches = match_map(None, expected_path)
+    else:
+        matches = match_map()
+
+    for ekey in expect.keys():
+        matches[ekey] = f'{expect[ekey]}="{expected_path}"'
+
+    run_test(runner, paths, args, matches.values(), cwd=str(paths.root))
 
 
-def match_map(yadm_dir=None):
+def match_map(yadm_dir=None, yadm_data=None):
     """Create a dictionary of matches, relative to yadm_dir"""
     if not yadm_dir:
-        yadm_dir = '/'.join([HOME, YDIR])
+        yadm_dir = "/".join([HOME, YDIR])
+    if not yadm_data:
+        yadm_data = "/".join([HOME, YDATA])
     return {
-        'yadm': f'YADM_DIR="{yadm_dir}"',
-        'repo': f'YADM_REPO="{yadm_dir}/{REPO}"',
-        'config': f'YADM_CONFIG="{yadm_dir}/{CONFIG}"',
-        'encrypt': f'YADM_ENCRYPT="{yadm_dir}/{ENCRYPT}"',
-        'archive': f'YADM_ARCHIVE="{yadm_dir}/{ARCHIVE}"',
-        'bootstrap': f'YADM_BOOTSTRAP="{yadm_dir}/{BOOTSTRAP}"',
-        'git': f'GIT_DIR="{yadm_dir}/{REPO}"',
-        }
+        "yadm": f'YADM_DIR="{yadm_dir}"',
+        "repo": f'YADM_REPO="{yadm_data}/{REPO}"',
+        "config": f'YADM_CONFIG="{yadm_dir}/{CONFIG}"',
+        "encrypt": f'YADM_ENCRYPT="{yadm_dir}/{ENCRYPT}"',
+        "archive": f'YADM_ARCHIVE="{yadm_data}/{ARCHIVE}"',
+        "bootstrap": f'YADM_BOOTSTRAP="{yadm_dir}/{BOOTSTRAP}"',
+        "git": f'GIT_DIR="{yadm_data}/{REPO}"',
+    }
 
 
-def run_test(runner, paths, args, expected_matches, expected_code=0):
+def run_test(runner, paths, args, expected_matches, cwd=None):
     """Run proces global args, and run configure_paths"""
-    argstring = ' '.join(['"'+a+'"' for a in args])
+    argstring = " ".join(['"' + a + '"' for a in args])
     script = f"""
         YADM_TEST=1 HOME="{HOME}" source {paths.pgm}
         process_global_args {argstring}
-        HOME="{HOME}" set_yadm_dir
+        XDG_CONFIG_HOME=
+        XDG_DATA_HOME=
+        HOME="{HOME}" set_yadm_dirs
         configure_paths
         declare -p | grep -E '(YADM|GIT)_'
     """
-    run = runner(command=['bash'], inp=script)
-    assert run.code == expected_code
-    assert run.err == ''
+    run = runner(command=["bash"], inp=script, cwd=cwd)
+    assert run.success
+    assert run.err == ""
     for match in expected_matches:
         assert match in run.out

test/test_unit_copy_perms.py

@@ -0,0 +1,52 @@
+"""Unit tests: copy_perms"""
+import os
+
+import pytest
+
+OCTAL = "7654"
+NON_OCTAL = "9876"
+
+
+@pytest.mark.parametrize("stat_broken", [True, False], ids=["normal", "stat broken"])
+def test_copy_perms(runner, yadm, tmpdir, stat_broken):
+    """Test function copy_perms"""
+    src_mode = 0o754
+    dst_mode = 0o644
+    source = tmpdir.join("source")
+    source.write("test", ensure=True)
+    source.chmod(src_mode)
+
+    dest = tmpdir.join("dest")
+    dest.write("test", ensure=True)
+    dest.chmod(dst_mode)
+
+    override_stat = ""
+    if stat_broken:
+        override_stat = "function stat() { echo broken; }"
+    script = f"""
+        YADM_TEST=1 source {yadm}
+        {override_stat}
+        copy_perms "{source}" "{dest}"
+    """
+    run = runner(command=["bash"], inp=script)
+    assert run.success
+    assert run.err == ""
+    assert run.out == ""
+    expected = dst_mode if stat_broken else src_mode
+    assert oct(os.stat(dest).st_mode)[-3:] == oct(expected)[-3:]
+
+
+@pytest.mark.parametrize("stat_output", [OCTAL, NON_OCTAL], ids=["octal", "non-octal"])
+def test_get_mode(runner, yadm, stat_output):
+    """Test function get_mode"""
+    script = f"""
+        YADM_TEST=1 source {yadm}
+        function stat() {{ echo {stat_output}; }}
+        mode=$(get_mode abc)
+        echo "MODE:$mode"
+    """
+    run = runner(command=["bash"], inp=script)
+    assert run.success
+    assert run.err == ""
+    expected = OCTAL if stat_output == OCTAL else ""
+    assert f"MODE:{expected}\n" in run.out

test/test_unit_encryption.py

@@ -0,0 +1,135 @@
+"""Unit tests: encryption functions"""
+
+import pytest
+
+
+@pytest.mark.parametrize("condition", ["default", "override"])
+def test_get_cipher(runner, paths, condition):
+    """Test _get_cipher()"""
+
+    if condition == "override":
+        paths.config.write("[yadm]\n\tcipher = override-cipher")
+
+    script = f"""
+        YADM_TEST=1 source {paths.pgm}
+        YADM_DIR="{paths.yadm}"
+        set_yadm_dirs
+        configure_paths
+        _get_cipher test-archive
+        echo "output_archive:$output_archive"
+        echo "yadm_cipher:$yadm_cipher"
+    """
+    run = runner(command=["bash"], inp=script)
+    assert run.success
+    assert run.err == ""
+    assert "output_archive:test-archive" in run.out
+    if condition == "override":
+        assert "yadm_cipher:override-cipher" in run.out
+    else:
+        assert "yadm_cipher:gpg" in run.out
+
+
+@pytest.mark.parametrize("cipher", ["gpg", "openssl", "bad"])
+@pytest.mark.parametrize("mode", ["_encrypt_to", "_decrypt_from"])
+def test_encrypt_decrypt(runner, paths, cipher, mode):
+    """Test _encrypt_to() & _decrypt_from"""
+
+    script = f"""
+        YADM_TEST=1 source {paths.pgm}
+        YADM_DIR="{paths.yadm}"
+        set_yadm_dirs
+        configure_paths
+        function mock_openssl() {{ echo openssl $*; }}
+        function mock_gpg() {{ echo gpg $*; }}
+        function _get_cipher() {{
+            output_archive="$1"
+            yadm_cipher="{cipher}"
+        }}
+        OPENSSL_PROGRAM=mock_openssl
+        GPG_PROGRAM=mock_gpg
+        {mode} {paths.archive}
+    """
+    run = runner(command=["bash"], inp=script)
+
+    if cipher != "bad":
+        assert run.success
+        assert run.out.startswith(cipher)
+        assert str(paths.archive) in run.out
+        assert run.err == ""
+    else:
+        assert run.failure
+        assert "Unknown cipher" in run.err
+
+
+@pytest.mark.parametrize("condition", ["default", "override"])
+def test_get_openssl_ciphername(runner, paths, condition):
+    """Test _get_openssl_ciphername()"""
+
+    if condition == "override":
+        paths.config.write("[yadm]\n\topenssl-ciphername = override-cipher")
+
+    script = f"""
+        YADM_TEST=1 source {paths.pgm}
+        YADM_DIR="{paths.yadm}"
+        set_yadm_dirs
+        configure_paths
+        result=$(_get_openssl_ciphername)
+        echo "result:$result"
+    """
+    run = runner(command=["bash"], inp=script)
+    assert run.success
+    assert run.err == ""
+    if condition == "override":
+        assert run.out.strip() == "result:override-cipher"
+    else:
+        assert run.out.strip() == "result:aes-256-cbc"
+
+
+@pytest.mark.parametrize("condition", ["old", "not-old"])
+def test_set_openssl_options(runner, paths, condition):
+    """Test _set_openssl_options()"""
+
+    if condition == "old":
+        paths.config.write("[yadm]\n\topenssl-old = true")
+
+    script = f"""
+        YADM_TEST=1 source {paths.pgm}
+        YADM_DIR="{paths.yadm}"
+        set_yadm_dirs
+        configure_paths
+        function _get_openssl_ciphername() {{ echo "testcipher"; }}
+        _set_openssl_options
+        echo "result:${{OPENSSL_OPTS[@]}}"
+    """
+    run = runner(command=["bash"], inp=script)
+    assert run.success
+    assert run.err == ""
+    if condition == "old":
+        assert "-testcipher -salt -md md5" in run.out
+    else:
+        assert "-testcipher -salt -pbkdf2 -iter 100000 -md sha512" in run.out
+
+
+@pytest.mark.parametrize("recipient", ["ASK", "present", ""])
+def test_set_gpg_options(runner, paths, recipient):
+    """Test _set_gpg_options()"""
+
+    paths.config.write(f"[yadm]\n\tgpg-recipient = {recipient}")
+
+    script = f"""
+        YADM_TEST=1 source {paths.pgm}
+        YADM_DIR="{paths.yadm}"
+        set_yadm_dirs
+        configure_paths
+        _set_gpg_options
+        echo "result:${{GPG_OPTS[@]}}"
+    """
+    run = runner(command=["bash"], inp=script)
+    assert run.success
+    assert run.err == ""
+    if recipient == "ASK":
+        assert run.out.strip() == "result:--no-default-recipient -e"
+    elif recipient != "":
+        assert run.out.strip() == f"result:-e -r {recipient}"
+    else:
+        assert run.out.strip() == "result:-c"

test/test_unit_exclude_encrypted.py

@@ -2,38 +2,28 @@
 import pytest
 
 
-@pytest.mark.parametrize(
-    'exclude', ['missing', 'outdated', 'up-to-date'])
-@pytest.mark.parametrize(
-    'encrypt_exists', [True, False], ids=['encrypt', 'no-encrypt'])
-@pytest.mark.parametrize(
-    'auto_exclude', [True, False], ids=['enabled', 'disabled'])
-def test_exclude_encrypted(
-        runner, tmpdir, yadm, encrypt_exists, auto_exclude, exclude):
+@pytest.mark.parametrize("exclude", ["missing", "outdated", "up-to-date"])
+@pytest.mark.parametrize("encrypt_exists", [True, False], ids=["encrypt", "no-encrypt"])
+@pytest.mark.parametrize("auto_exclude", [True, False], ids=["enabled", "disabled"])
+def test_exclude_encrypted(runner, tmpdir, yadm, encrypt_exists, auto_exclude, exclude):
     """Test exclude_encrypted()"""
 
-    header = (
-        "# yadm-auto-excludes\n"
-        "# This section is managed by yadm.\n"
-        "# Any edits below will be lost.\n"
-    )
+    header = "# yadm-auto-excludes\n# This section is managed by yadm.\n# Any edits below will be lost.\n"
 
     config_function = 'function config() { echo "false";}'
     if auto_exclude:
-        config_function = 'function config() { return; }'
+        config_function = "function config() { return; }"
 
-    encrypt_file = tmpdir.join('encrypt_file')
-    repo_dir = tmpdir.join('repodir')
-    exclude_file = repo_dir.join('info/exclude')
+    encrypt_file = tmpdir.join("encrypt_file")
+    repo_dir = tmpdir.join("repodir")
+    exclude_file = repo_dir.join("info/exclude")
 
     if encrypt_exists:
-        encrypt_file.write('test-encrypt-data\n', ensure=True)
-    if exclude == 'outdated':
-        exclude_file.write(
-            f'original-exclude\n{header}outdated\n', ensure=True)
-    elif exclude == 'up-to-date':
-        exclude_file.write(
-            f'original-exclude\n{header}test-encrypt-data\n', ensure=True)
+        encrypt_file.write("test-encrypt-data\n", ensure=True)
+    if exclude == "outdated":
+        exclude_file.write(f"original-exclude\n{header}outdated\n", ensure=True)
+    elif exclude == "up-to-date":
+        exclude_file.write(f"original-exclude\n{header}test-encrypt-data\n", ensure=True)
 
     script = f"""
         YADM_TEST=1 source {yadm}
@@ -43,24 +33,22 @@ def test_exclude_encrypted(
         YADM_REPO="{repo_dir}"
         exclude_encrypted
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
 
     if auto_exclude:
         if encrypt_exists:
             assert exclude_file.exists()
-            if exclude == 'missing':
-                assert exclude_file.read() == f'{header}test-encrypt-data\n'
+            if exclude == "missing":
+                assert exclude_file.read() == f"{header}test-encrypt-data\n"
             else:
-                assert exclude_file.read() == (
-                    'original-exclude\n'
-                    f'{header}test-encrypt-data\n')
-            if exclude != 'up-to-date':
-                assert f'Updating {exclude_file}' in run.out
+                assert exclude_file.read() == ("original-exclude\n" f"{header}test-encrypt-data\n")
+            if exclude != "up-to-date":
+                assert f"Updating {exclude_file}" in run.out
             else:
-                assert run.out == ''
+                assert run.out == ""
         else:
-            assert run.out == ''
+            assert run.out == ""
     else:
-        assert run.out == ''
+        assert run.out == ""

test/test_unit_is_valid_branch_name.py

@@ -1,40 +0,0 @@
-"""Unit tests: is_valid_branch_name"""
-import pytest
-
-# Git branches do not allow:
-#  * path component that begins with "."
-#  * double dot
-#  * "~", "^", ":", "\", space
-#  * end with a "/"
-#  * end with ".lock"
-
-
-@pytest.mark.parametrize(
-    'branch, expected', [
-        ('master', 'valid'),
-        ('path/branch', 'valid'),
-        ('path/.branch', 'invalid'),
-        ('path..branch', 'invalid'),
-        ('path~branch', 'invalid'),
-        ('path^branch', 'invalid'),
-        ('path:branch', 'invalid'),
-        ('path\\branch', 'invalid'),
-        ('path branch', 'invalid'),
-        ('path/branch/', 'invalid'),
-        ('branch.lock', 'invalid'),
-    ])
-def test_is_valid_branch_name(runner, yadm, branch, expected):
-    """Test function is_valid_branch_name()"""
-
-    script = f"""
-        YADM_TEST=1 source {yadm}
-        if is_valid_branch_name "{branch}"; then
-            echo valid
-        else
-            echo invalid
-        fi
-    """
-    run = runner(command=['bash'], inp=script)
-    assert run.success
-    assert run.err == ''
-    assert run.out.strip() == expected

test/test_unit_issue_legacy_path_warning.py

@@ -3,37 +3,38 @@ import pytest
 
 
 @pytest.mark.parametrize(
-    'legacy_path', [
+    "legacy_path",
+    [
         None,
-        'repo.git',
-        'config',
-        'encrypt',
-        'files.gpg',
-        'bootstrap',
-        'hooks/pre_command',
-        'hooks/post_command',
-        ],
-    )
-@pytest.mark.parametrize(
-    'upgrade', [True, False], ids=['upgrade', 'no-upgrade'])
-def test_legacy_warning(tmpdir, runner, yadm, upgrade, legacy_path):
+        "repo.git",
+        "files.gpg",
+    ],
+)
+@pytest.mark.parametrize("override", [True, False], ids=["override", "no-override"])
+@pytest.mark.parametrize("upgrade", [True, False], ids=["upgrade", "no-upgrade"])
+def test_legacy_warning(tmpdir, runner, yadm, upgrade, override, legacy_path):
     """Use issue_legacy_path_warning"""
-    home = tmpdir.mkdir('home')
+    home = tmpdir.mkdir("home")
 
     if legacy_path:
-        home.mkdir(f'.yadm').ensure(legacy_path)
+        home.ensure(f".config/yadm/{str(legacy_path)}")
 
-    main_args = 'MAIN_ARGS=("upgrade")' if upgrade else ''
+    override = "YADM_OVERRIDE_REPO=override" if override else ""
+    main_args = 'MAIN_ARGS=("upgrade")' if upgrade else ""
     script = f"""
+        XDG_CONFIG_HOME=
+        XDG_DATA_HOME=
         HOME={home}
         YADM_TEST=1 source {yadm}
         {main_args}
+        {override}
+        set_yadm_dirs
         issue_legacy_path_warning
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
-    if legacy_path and not upgrade:
-        assert 'Legacy configuration paths have been detected' in run.out
+    assert run.out == ""
+    if legacy_path and (not upgrade) and (not override):
+        assert "Legacy paths have been detected" in run.err
     else:
-        assert run.out.rstrip() == ''
+        assert "Legacy paths have been detected" not in run.err

test/test_unit_parse_encrypt.py

@@ -7,165 +7,184 @@ def test_not_called(runner, paths):
     """Test parse_encrypt (not called)"""
     run = run_parse_encrypt(runner, paths, skip_parse=True)
     assert run.success
-    assert run.err == ''
-    assert 'EIF:unparsed' in run.out, 'EIF should be unparsed'
-    assert 'EIF_COUNT:1' in run.out, 'Only value of EIF should be unparsed'
+    assert run.err == ""
+    assert "EIF:unparsed" in run.out, "EIF should be unparsed"
+    assert "EIF_COUNT:1" in run.out, "Only value of EIF should be unparsed"
 
 
 def test_short_circuit(runner, paths):
     """Test parse_encrypt (short-circuit)"""
     run = run_parse_encrypt(runner, paths, twice=True)
     assert run.success
-    assert run.err == ''
-    assert 'PARSE_ENCRYPT_SHORT=parse_encrypt() not reprocessed' in run.out, (
-        'parse_encrypt() should short-circuit')
+    assert run.err == ""
+    assert "PARSE_ENCRYPT_SHORT=parse_encrypt() not reprocessed" in run.out, "parse_encrypt() should short-circuit"
 
 
 @pytest.mark.parametrize(
-    'encrypt', [
-        ('missing'),
-        ('empty'),
-    ])
+    "encrypt",
+    [
+        ("missing"),
+        ("empty"),
+    ],
+)
 def test_empty(runner, paths, encrypt):
     """Test parse_encrypt (file missing/empty)"""
 
     # write encrypt file
-    if encrypt == 'missing':
-        assert not paths.encrypt.exists(), 'Encrypt should be missing'
+    if encrypt == "missing":
+        assert not paths.encrypt.exists(), "Encrypt should be missing"
     else:
-        paths.encrypt.write('')
-        assert paths.encrypt.exists(), 'Encrypt should exist'
-        assert paths.encrypt.size() == 0, 'Encrypt should be empty'
+        paths.encrypt.write("")
+        assert paths.encrypt.exists(), "Encrypt should exist"
+        assert paths.encrypt.size() == 0, "Encrypt should be empty"
 
     # run parse_encrypt
     run = run_parse_encrypt(runner, paths)
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
 
     # validate parsing result
-    assert 'EIF_COUNT:0' in run.out, 'EIF should be empty'
+    assert "EIF_COUNT:0" in run.out, "EIF should be empty"
 
 
-@pytest.mark.usefixtures('ds1_repo_copy')
+def create_test_encrypt_data(paths):
+    """Generate test data for testing encrypt"""
+
+    edata = ""
+    expected = set()
+
+    # empty line
+    edata += "\n"
+
+    # simple comments
+    edata += "# a simple comment\n"
+    edata += "    # a comment with leading space\n"
+
+    # unreferenced directory
+    paths.work.join("unreferenced").mkdir()
+
+    # simple files
+    edata += "simple_file\n"
+    edata += "simple.file\n"
+    paths.work.join("simple_file").write("")
+    paths.work.join("simple.file").write("")
+    paths.work.join("simple_file2").write("")
+    paths.work.join("simple.file2").write("")
+    expected.add("simple_file")
+    expected.add("simple.file")
+
+    # simple files in directories
+    edata += "simple_dir/simple_file\n"
+    paths.work.join("simple_dir/simple_file").write("", ensure=True)
+    paths.work.join("simple_dir/simple_file2").write("", ensure=True)
+    expected.add("simple_dir/simple_file")
+
+    # paths with spaces
+    edata += "with space/with space\n"
+    paths.work.join("with space/with space").write("", ensure=True)
+    paths.work.join("with space/with space2").write("", ensure=True)
+    expected.add("with space/with space")
+
+    # hidden files
+    edata += ".hidden\n"
+    paths.work.join(".hidden").write("")
+    expected.add(".hidden")
+
+    # hidden files in directories
+    edata += ".hidden_dir/.hidden_file\n"
+    paths.work.join(".hidden_dir/.hidden_file").write("", ensure=True)
+    expected.add(".hidden_dir/.hidden_file")
+
+    # wildcards
+    edata += "wild*\n"
+    paths.work.join("wildcard1").write("", ensure=True)
+    paths.work.join("wildcard2").write("", ensure=True)
+    expected.add("wildcard1")
+    expected.add("wildcard2")
+
+    edata += "dirwild*\n"
+    paths.work.join("dirwildcard/file1").write("", ensure=True)
+    paths.work.join("dirwildcard/file2").write("", ensure=True)
+    expected.add("dirwildcard")
+
+    # excludes
+    edata += "exclude*\n"
+    edata += "ex ex/*\n"
+    paths.work.join("exclude_file1").write("")
+    paths.work.join("exclude_file2.ex").write("")
+    paths.work.join("exclude_file3.ex3").write("")
+    expected.add("exclude_file1")
+    expected.add("exclude_file3.ex3")
+    edata += "!*.ex\n"
+    edata += "!ex ex/*.txt\n"
+    paths.work.join("ex ex/file4").write("", ensure=True)
+    paths.work.join("ex ex/file5.txt").write("", ensure=True)
+    paths.work.join("ex ex/file6.text").write("", ensure=True)
+    expected.add("ex ex/file4")
+    expected.add("ex ex/file6.text")
+
+    # double star
+    edata += "doublestar/**/file*\n"
+    edata += "!**/file3\n"
+    paths.work.join("doublestar/a/b/file1").write("", ensure=True)
+    paths.work.join("doublestar/c/d/file2").write("", ensure=True)
+    paths.work.join("doublestar/e/f/file3").write("", ensure=True)
+    paths.work.join("doublestar/g/h/nomatch").write("", ensure=True)
+    expected.add("doublestar/a/b/file1")
+    expected.add("doublestar/c/d/file2")
+    # doublestar/e/f/file3 is excluded
+
+    return edata, expected
+
+
+@pytest.mark.usefixtures("ds1_repo_copy")
 def test_file_parse_encrypt(runner, paths):
     """Test parse_encrypt
 
     Test an array of supported features of the encrypt configuration.
     """
 
-    edata = ''
-    expected = set()
-
-    # empty line
-    edata += '\n'
-
-    # simple comments
-    edata += '# a simple comment\n'
-    edata += '    # a comment with leading space\n'
-
-    # unreferenced directory
-    paths.work.join('unreferenced').mkdir()
-
-    # simple files
-    edata += 'simple_file\n'
-    edata += 'simple.file\n'
-    paths.work.join('simple_file').write('')
-    paths.work.join('simple.file').write('')
-    paths.work.join('simple_file2').write('')
-    paths.work.join('simple.file2').write('')
-    expected.add('simple_file')
-    expected.add('simple.file')
-
-    # simple files in directories
-    edata += 'simple_dir/simple_file\n'
-    paths.work.join('simple_dir/simple_file').write('', ensure=True)
-    paths.work.join('simple_dir/simple_file2').write('', ensure=True)
-    expected.add('simple_dir/simple_file')
-
-    # paths with spaces
-    edata += 'with space/with space\n'
-    paths.work.join('with space/with space').write('', ensure=True)
-    paths.work.join('with space/with space2').write('', ensure=True)
-    expected.add('with space/with space')
-
-    # hidden files
-    edata += '.hidden\n'
-    paths.work.join('.hidden').write('')
-    expected.add('.hidden')
-
-    # hidden files in directories
-    edata += '.hidden_dir/.hidden_file\n'
-    paths.work.join('.hidden_dir/.hidden_file').write('', ensure=True)
-    expected.add('.hidden_dir/.hidden_file')
-
-    # wildcards
-    edata += 'wild*\n'
-    paths.work.join('wildcard1').write('', ensure=True)
-    paths.work.join('wildcard2').write('', ensure=True)
-    expected.add('wildcard1')
-    expected.add('wildcard2')
-
-    edata += 'dirwild*\n'
-    paths.work.join('dirwildcard/file1').write('', ensure=True)
-    paths.work.join('dirwildcard/file2').write('', ensure=True)
-    expected.add('dirwildcard')
-
-    # excludes
-    edata += 'exclude*\n'
-    edata += 'ex ex/*\n'
-    paths.work.join('exclude_file1').write('')
-    paths.work.join('exclude_file2.ex').write('')
-    paths.work.join('exclude_file3.ex3').write('')
-    expected.add('exclude_file1')
-    expected.add('exclude_file3.ex3')
-    edata += '!*.ex\n'
-    edata += '!ex ex/*.txt\n'
-    paths.work.join('ex ex/file4').write('', ensure=True)
-    paths.work.join('ex ex/file5.txt').write('', ensure=True)
-    paths.work.join('ex ex/file6.text').write('', ensure=True)
-    expected.add('ex ex/file4')
-    expected.add('ex ex/file6.text')
+    # generate test data & expectations
+    edata, expected = create_test_encrypt_data(paths)
 
     # write encrypt file
-    print(f'ENCRYPT:\n---\n{edata}---\n')
+    print(f"ENCRYPT:\n---\n{edata}---\n")
     paths.encrypt.write(edata)
     assert paths.encrypt.isfile()
 
     # run parse_encrypt
     run = run_parse_encrypt(runner, paths)
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
 
-    assert f'EIF_COUNT:{len(expected)}' in run.out, 'EIF count wrong'
+    assert f"EIF_COUNT:{len(expected)}" in run.out, "EIF count wrong"
     for expected_file in expected:
-        assert f'EIF:{expected_file}\n' in run.out
+        assert f"EIF:{expected_file}\n" in run.out
 
-    sorted_expectations = '\n'.join(
-        [f'EIF:{exp}' for exp in sorted(expected)])
+    sorted_expectations = "\n".join([f"EIF:{exp}" for exp in sorted(expected)])
     assert sorted_expectations in run.out
 
 
-def run_parse_encrypt(
-        runner, paths,
-        skip_parse=False,
-        twice=False):
+def run_parse_encrypt(runner, paths, skip_parse=False, twice=False):
     """Run parse_encrypt
 
     A count of ENCRYPT_INCLUDE_FILES will be reported as EIF_COUNT:X. All
     values of ENCRYPT_INCLUDE_FILES will be reported as individual EIF:value
     lines.
     """
-    parse_cmd = 'parse_encrypt'
+    parse_cmd = "parse_encrypt"
     if skip_parse:
-        parse_cmd = ''
+        parse_cmd = ""
     if twice:
-        parse_cmd = 'parse_encrypt; parse_encrypt'
+        parse_cmd = "parse_encrypt; parse_encrypt"
     script = f"""
         YADM_TEST=1 source {paths.pgm}
         YADM_ENCRYPT={paths.encrypt}
         export YADM_ENCRYPT
         GIT_DIR={paths.repo}
         export GIT_DIR
+        YADM_WORK={paths.work}
+        export YADM_WORK
         {parse_cmd}
         export ENCRYPT_INCLUDE_FILES
         export PARSE_ENCRYPT_SHORT
@@ -175,5 +194,5 @@ def run_parse_encrypt(
             echo "EIF:$value"
         done
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     return run

test/test_unit_private_dirs.py

@@ -0,0 +1,34 @@
+"""Unit tests: private_dirs"""
+import pytest
+
+
+@pytest.mark.parametrize(
+    "gnupghome",
+    [True, False],
+    ids=["gnupghome-set", "gnupghome-unset"],
+)
+@pytest.mark.parametrize("param", ["all", "gnupg"])
+def test_relative_path(runner, paths, gnupghome, param):
+    """Test translate_to_relative"""
+
+    alt_gnupghome = "alt/gnupghome"
+    env_gnupghome = paths.work.join(alt_gnupghome)
+
+    script = f"""
+        YADM_TEST=1 source {paths.pgm}
+        YADM_WORK={paths.work}
+        private_dirs {param}
+    """
+
+    env = {}
+    if gnupghome:
+        env["GNUPGHOME"] = env_gnupghome
+
+    expected = alt_gnupghome if gnupghome else ".gnupg"
+    if param == "all":
+        expected = f".ssh {expected}"
+
+    run = runner(command=["bash"], inp=script, env=env)
+    assert run.success
+    assert run.err == ""
+    assert run.out.strip() == expected

test/test_unit_query_distro.py

@@ -1,26 +1,30 @@
 """Unit tests: query_distro"""
+import pytest
 
 
-def test_lsb_release_present(runner, yadm, tst_distro):
+@pytest.mark.parametrize("condition", ["lsb_release", "os-release", "os-release-quotes", "missing"])
+def test_query_distro(runner, yadm, tst_distro, tmp_path, condition):
     """Match lsb_release -si when present"""
+    test_release = "testrelease"
+    lsb_release = ""
+    os_release = tmp_path.joinpath("os-release")
+    if "os-release" in condition:
+        quotes = '"' if "quotes" in condition else ""
+        os_release.write_text(f"testing\nID={quotes}{test_release}{quotes}\nrelease")
+    if condition != "lsb_release":
+        lsb_release = 'LSB_RELEASE_PROGRAM="missing_lsb_release"'
     script = f"""
         YADM_TEST=1 source {yadm}
+        {lsb_release}
+        OS_RELEASE="{os_release}"
         query_distro
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
-    assert run.out.rstrip() == tst_distro
-
-
-def test_lsb_release_missing(runner, yadm):
-    """Empty value when missing"""
-    script = f"""
-        YADM_TEST=1 source {yadm}
-        LSB_RELEASE_PROGRAM="missing_lsb_release"
-        query_distro
-    """
-    run = runner(command=['bash'], inp=script)
-    assert run.success
-    assert run.err == ''
-    assert run.out.rstrip() == ''
+    assert run.err == ""
+    if condition == "lsb_release":
+        assert run.out.rstrip() == tst_distro
+    elif "os-release" in condition:
+        assert run.out.rstrip() == test_release
+    else:
+        assert run.out.rstrip() == ""

test/test_unit_query_distro_family.py

@@ -0,0 +1,24 @@
+"""Unit tests: query_distro_family"""
+import pytest
+
+
+@pytest.mark.parametrize("condition", ["os-release", "os-release-quotes", "missing"])
+def test_query_distro_family(runner, yadm, tmp_path, condition):
+    """Match ID_LIKE when present"""
+    test_family = "testfamily"
+    os_release = tmp_path.joinpath("os-release")
+    if "os-release" in condition:
+        quotes = '"' if "quotes" in condition else ""
+        os_release.write_text(f"testing\nID_LIKE={quotes}{test_family}{quotes}\nfamily")
+    script = f"""
+        YADM_TEST=1 source {yadm}
+        OS_RELEASE="{os_release}"
+        query_distro_family
+    """
+    run = runner(command=["bash"], inp=script)
+    assert run.success
+    assert run.err == ""
+    if "os-release" in condition:
+        assert run.out.rstrip() == test_family
+    else:
+        assert run.out.rstrip() == ""

test/test_unit_record_score.py

@@ -8,16 +8,16 @@ INIT_VARS = """
     local_host=testhost
     local_user=testuser
     alt_scores=()
-    alt_filenames=()
     alt_targets=()
+    alt_sources=()
     alt_template_cmds=()
 """
 
 REPORT_RESULTS = """
     echo "SIZE:${#alt_scores[@]}"
     echo "SCORES:${alt_scores[@]}"
-    echo "FILENAMES:${alt_filenames[@]}"
     echo "TARGETS:${alt_targets[@]}"
+    echo "SOURCES:${alt_sources[@]}"
 """
 
 
@@ -27,16 +27,16 @@ def test_dont_record_zeros(runner, yadm):
     script = f"""
         YADM_TEST=1 source {yadm}
         {INIT_VARS}
-        record_score "0" "testfile" "testtarget"
+        record_score "0" "testtgt" "testsrc"
         {REPORT_RESULTS}
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
-    assert 'SIZE:0\n' in run.out
-    assert 'SCORES:\n' in run.out
-    assert 'FILENAMES:\n' in run.out
-    assert 'TARGETS:\n' in run.out
+    assert run.err == ""
+    assert "SIZE:0\n" in run.out
+    assert "SCORES:\n" in run.out
+    assert "TARGETS:\n" in run.out
+    assert "SOURCES:\n" in run.out
 
 
 def test_new_scores(runner, yadm):
@@ -45,70 +45,97 @@ def test_new_scores(runner, yadm):
     script = f"""
         YADM_TEST=1 source {yadm}
         {INIT_VARS}
-        record_score "1" "file_one"   "targ_one"
-        record_score "2" "file_two"   "targ_two"
-        record_score "4" "file_three" "targ_three"
+        record_score "1" "tgt_one"   "src_one"
+        record_score "2" "tgt_two"   "src_two"
+        record_score "4" "tgt_three" "src_three"
         {REPORT_RESULTS}
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
-    assert 'SIZE:3\n' in run.out
-    assert 'SCORES:1 2 4\n' in run.out
-    assert 'FILENAMES:file_one file_two file_three\n' in run.out
-    assert 'TARGETS:targ_one targ_two targ_three\n' in run.out
+    assert run.err == ""
+    assert "SIZE:3\n" in run.out
+    assert "SCORES:1 2 4\n" in run.out
+    assert "TARGETS:tgt_one tgt_two tgt_three\n" in run.out
+    assert "SOURCES:src_one src_two src_three\n" in run.out
 
 
-@pytest.mark.parametrize('difference', ['lower', 'equal', 'higher'])
+@pytest.mark.parametrize("difference", ["lower", "equal", "higher"])
 def test_existing_scores(runner, yadm, difference):
     """Test existing scores"""
 
-    expected_score = '2'
-    expected_target = 'existing_target'
-    if difference == 'lower':
-        score = '1'
-    elif difference == 'equal':
-        score = '2'
+    expected_score = "2"
+    expected_src = "existing_src"
+    if difference == "lower":
+        score = "1"
+    elif difference == "equal":
+        score = "2"
     else:
-        score = '4'
-        expected_score = '4'
-        expected_target = 'new_target'
+        score = "4"
+        expected_score = "4"
+        expected_src = "new_src"
 
     script = f"""
         YADM_TEST=1 source {yadm}
         {INIT_VARS}
         alt_scores=(2)
-        alt_filenames=("testfile")
-        alt_targets=("existing_target")
-        record_score "{score}" "testfile" "new_target"
+        alt_targets=("testtgt")
+        alt_sources=("existing_src")
+        record_score "{score}" "testtgt" "new_src"
         {REPORT_RESULTS}
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
-    assert 'SIZE:1\n' in run.out
-    assert f'SCORES:{expected_score}\n' in run.out
-    assert 'FILENAMES:testfile\n' in run.out
-    assert f'TARGETS:{expected_target}\n' in run.out
+    assert run.err == ""
+    assert "SIZE:1\n" in run.out
+    assert f"SCORES:{expected_score}\n" in run.out
+    assert "TARGETS:testtgt\n" in run.out
+    assert f"SOURCES:{expected_src}\n" in run.out
 
 
 def test_existing_template(runner, yadm):
-    """Record nothing if a template command is registered for this file"""
+    """Record nothing if a template command is registered for this target"""
 
     script = f"""
         YADM_TEST=1 source {yadm}
         {INIT_VARS}
         alt_scores=(1)
-        alt_filenames=("testfile")
-        alt_targets=()
+        alt_targets=("testtgt")
+        alt_sources=()
         alt_template_cmds=("existing_template")
-        record_score "2" "testfile" "new_target"
+        record_score "2" "testtgt" "new_src"
         {REPORT_RESULTS}
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
-    assert 'SIZE:1\n' in run.out
-    assert 'SCORES:1\n' in run.out
-    assert 'FILENAMES:testfile\n' in run.out
-    assert 'TARGETS:\n' in run.out
+    assert run.err == ""
+    assert "SIZE:1\n" in run.out
+    assert "SCORES:1\n" in run.out
+    assert "TARGETS:testtgt\n" in run.out
+    assert "SOURCES:\n" in run.out
+
+
+def test_config_first(runner, yadm):
+    """Verify YADM_CONFIG is always processed first"""
+
+    config = "yadm_config_file"
+    script = f"""
+        YADM_TEST=1 source {yadm}
+        {INIT_VARS}
+        YADM_CONFIG={config}
+        record_score "1" "tgt_before" "src_before"
+        record_template "tgt_tmp" "cmd_tmp" "src_tmp"
+        record_score "2" "{config}"   "src_config"
+        record_score "3" "tgt_after"  "src_after"
+        {REPORT_RESULTS}
+        echo "CMD_VALUE:${{alt_template_cmds[@]}}"
+        echo "CMD_INDEX:${{!alt_template_cmds[@]}}"
+    """
+    run = runner(command=["bash"], inp=script)
+    assert run.success
+    assert run.err == ""
+    assert "SIZE:3\n" in run.out
+    assert "SCORES:2 1 3\n" in run.out
+    assert f"TARGETS:{config} tgt_before tgt_tmp tgt_after\n" in run.out
+    assert "SOURCES:src_config src_before src_tmp src_after\n" in run.out
+    assert "CMD_VALUE:cmd_tmp\n" in run.out
+    assert "CMD_INDEX:2\n" in run.out

test/test_unit_record_template.py

@@ -1,16 +1,16 @@
 """Unit tests: record_template"""
 
 INIT_VARS = """
-    alt_filenames=()
-    alt_template_cmds=()
     alt_targets=()
+    alt_template_cmds=()
+    alt_sources=()
 """
 
 REPORT_RESULTS = """
-    echo "SIZE:${#alt_filenames[@]}"
-    echo "FILENAMES:${alt_filenames[@]}"
+    echo "SIZE:${#alt_targets[@]}"
+    echo "TARGETS:${alt_targets[@]}"
     echo "CMDS:${alt_template_cmds[@]}"
-    echo "TARGS:${alt_targets[@]}"
+    echo "SOURCES:${alt_sources[@]}"
 """
 
 
@@ -20,18 +20,18 @@ def test_new_template(runner, yadm):
     script = f"""
         YADM_TEST=1 source {yadm}
         {INIT_VARS}
-        record_template "file_one"   "cmd_one"   "targ_one"
-        record_template "file_two"   "cmd_two"   "targ_two"
-        record_template "file_three" "cmd_three" "targ_three"
+        record_template "tgt_one"   "cmd_one"   "src_one"
+        record_template "tgt_two"   "cmd_two"   "src_two"
+        record_template "tgt_three" "cmd_three" "src_three"
         {REPORT_RESULTS}
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
-    assert 'SIZE:3\n' in run.out
-    assert 'FILENAMES:file_one file_two file_three\n' in run.out
-    assert 'CMDS:cmd_one cmd_two cmd_three\n' in run.out
-    assert 'TARGS:targ_one targ_two targ_three\n' in run.out
+    assert run.err == ""
+    assert "SIZE:3\n" in run.out
+    assert "TARGETS:tgt_one tgt_two tgt_three\n" in run.out
+    assert "CMDS:cmd_one cmd_two cmd_three\n" in run.out
+    assert "SOURCES:src_one src_two src_three\n" in run.out
 
 
 def test_existing_template(runner, yadm):
@@ -40,16 +40,16 @@ def test_existing_template(runner, yadm):
     script = f"""
         YADM_TEST=1 source {yadm}
         {INIT_VARS}
-        alt_filenames=("testfile")
+        alt_targets=("testtgt")
         alt_template_cmds=("existing_cmd")
-        alt_targets=("existing_targ")
-        record_template "testfile" "new_cmd" "new_targ"
+        alt_sources=("existing_src")
+        record_template "testtgt" "new_cmd" "new_src"
         {REPORT_RESULTS}
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
-    assert 'SIZE:1\n' in run.out
-    assert 'FILENAMES:testfile\n' in run.out
-    assert 'CMDS:new_cmd\n' in run.out
-    assert 'TARGS:new_targ\n' in run.out
+    assert run.err == ""
+    assert "SIZE:1\n" in run.out
+    assert "TARGETS:testtgt\n" in run.out
+    assert "CMDS:new_cmd\n" in run.out
+    assert "SOURCES:new_src\n" in run.out

test/test_unit_relative_path.py

@@ -0,0 +1,31 @@
+"""Unit tests: relative_path"""
+import pytest
+
+
+@pytest.mark.parametrize(
+    "base,full_path,expected",
+    [
+        ("/A/B/C", "/A", "../.."),
+        ("/A/B/C", "/A/B", ".."),
+        ("/A/B/C", "/A/B/C", ""),
+        ("/A/B/C", "/A/B/C/D", "D"),
+        ("/A/B/C", "/A/B/C/D/E", "D/E"),
+        ("/A/B/C", "/A/B/D", "../D"),
+        ("/A/B/C", "/A/B/D/E", "../D/E"),
+        ("/A/B/C", "/A/D", "../../D"),
+        ("/A/B/C", "/A/D/E", "../../D/E"),
+        ("/A/B/C", "/D/E/F", "../../../D/E/F"),
+    ],
+)
+def test_relative_path(runner, paths, base, full_path, expected):
+    """Test translate_to_relative"""
+
+    script = f"""
+        YADM_TEST=1 source {paths.pgm}
+        relative_path "{base}" "{full_path}"
+    """
+
+    run = runner(command=["bash"], inp=script)
+    assert run.success
+    assert run.err == ""
+    assert run.out.strip() == expected

test/test_unit_remove_stale_links.py

@@ -1,23 +1,24 @@
 """Unit tests: remove_stale_links"""
 import os
+
 import pytest
 
 
-@pytest.mark.parametrize('linked', [True, False])
-@pytest.mark.parametrize('kind', ['file', 'symlink'])
+@pytest.mark.parametrize("linked", [True, False])
+@pytest.mark.parametrize("kind", ["file", "symlink"])
 def test_remove_stale_links(runner, yadm, tmpdir, kind, linked):
     """Test remove_stale_links()"""
 
-    source_file = tmpdir.join('source_file')
-    source_file.write('source file', ensure=True)
-    link = tmpdir.join('link')
+    source_file = tmpdir.join("source_file")
+    source_file.write("source file", ensure=True)
+    link = tmpdir.join("link")
 
-    if kind == 'file':
-        link.write('link file', ensure=True)
+    if kind == "file":
+        link.write("link file", ensure=True)
     else:
-        os.system(f'ln -s {source_file} {link}')
+        os.system(f"ln -s {source_file} {link}")
 
-    alt_linked = ''
+    alt_linked = ""
     if linked:
         alt_linked = source_file
 
@@ -29,9 +30,9 @@ def test_remove_stale_links(runner, yadm, tmpdir, kind, linked):
         remove_stale_links
     """
 
-    run = runner(command=['bash'], inp=script)
-    assert run.err == ''
-    if kind == 'symlink' and not linked:
-        assert f'rm -f {link}' in run.out
+    run = runner(command=["bash"], inp=script)
+    assert run.err == ""
+    if kind == "symlink" and not linked:
+        assert f"rm -f {link}" in run.out
     else:
-        assert run.out == ''
+        assert run.out == ""

test/test_unit_report_invalid_alts.py

@@ -0,0 +1,30 @@
+"""Unit tests: report_invalid_alts"""
+import pytest
+
+
+@pytest.mark.parametrize("valid", [True, False], ids=["valid", "no_valid"])
+@pytest.mark.parametrize("previous", [True, False], ids=["prev", "no_prev"])
+def test_report_invalid_alts(runner, yadm, valid, previous):
+    """Use report_invalid_alts"""
+
+    lwi = ""
+    alts = "INVALID_ALT=()"
+    if previous:
+        lwi = "LEGACY_WARNING_ISSUED=1"
+    if not valid:
+        alts = 'INVALID_ALT=("file##invalid")'
+
+    script = f"""
+        YADM_TEST=1 source {yadm}
+        {lwi}
+        {alts}
+        report_invalid_alts
+    """
+    run = runner(command=["bash"], inp=script)
+    assert run.success
+    assert run.out == ""
+    if not valid and not previous:
+        assert "WARNING" in run.err
+        assert "file##invalid" in run.err
+    else:
+        assert run.err == ""

test/test_unit_score_file.py

@@ -2,32 +2,40 @@
 import pytest
 
 CONDITION = {
-    'default': {
-        'labels': ['default'],
-        'modifier': 0,
-        },
-    'system': {
-        'labels': ['o', 'os'],
-        'modifier': 1,
-        },
-    'distro': {
-        'labels': ['d', 'distro'],
-        'modifier': 2,
-        },
-    'class': {
-        'labels': ['c', 'class'],
-        'modifier': 4,
-        },
-    'hostname': {
-        'labels': ['h', 'hostname'],
-        'modifier': 8,
-        },
-    'user': {
-        'labels': ['u', 'user'],
-        'modifier': 16,
-        },
-    }
-TEMPLATE_LABELS = ['t', 'template', 'yadm']
+    "default": {
+        "labels": ["default"],
+        "modifier": 0,
+    },
+    "arch": {
+        "labels": ["a", "arch"],
+        "modifier": 1,
+    },
+    "system": {
+        "labels": ["o", "os"],
+        "modifier": 2,
+    },
+    "distro": {
+        "labels": ["d", "distro"],
+        "modifier": 4,
+    },
+    "distro_family": {
+        "labels": ["f", "distro_family"],
+        "modifier": 8,
+    },
+    "class": {
+        "labels": ["c", "class"],
+        "modifier": 16,
+    },
+    "hostname": {
+        "labels": ["h", "hostname"],
+        "modifier": 32,
+    },
+    "user": {
+        "labels": ["u", "user"],
+        "modifier": 64,
+    },
+}
+TEMPLATE_LABELS = ["t", "template", "yadm"]
 
 
 def calculate_score(filename):
@@ -35,42 +43,48 @@ def calculate_score(filename):
     # pylint: disable=too-many-branches
     score = 0
 
-    _, conditions = filename.split('##', 1)
+    _, conditions = filename.split("##", 1)
 
-    for condition in conditions.split(','):
+    for condition in conditions.split(","):
         label = condition
         value = None
-        if '.' in condition:
-            label, value = condition.split('.', 1)
-        if label in CONDITION['default']['labels']:
+        if "." in condition:
+            label, value = condition.split(".", 1)
+        if label in CONDITION["default"]["labels"]:
             score += 1000
-        elif label in CONDITION['system']['labels']:
-            if value == 'testsystem':
-                score += 1000 + CONDITION['system']['modifier']
+        elif label in CONDITION["arch"]["labels"]:
+            if value == "testarch":
+                score += 1000 + CONDITION["arch"]["modifier"]
             else:
                 score = 0
                 break
-        elif label in CONDITION['distro']['labels']:
-            if value == 'testdistro':
-                score += 1000 + CONDITION['distro']['modifier']
+        elif label in CONDITION["system"]["labels"]:
+            if value == "testsystem":
+                score += 1000 + CONDITION["system"]["modifier"]
             else:
                 score = 0
                 break
-        elif label in CONDITION['class']['labels']:
-            if value == 'testclass':
-                score += 1000 + CONDITION['class']['modifier']
+        elif label in CONDITION["distro"]["labels"]:
+            if value == "testdistro":
+                score += 1000 + CONDITION["distro"]["modifier"]
             else:
                 score = 0
                 break
-        elif label in CONDITION['hostname']['labels']:
-            if value == 'testhost':
-                score += 1000 + CONDITION['hostname']['modifier']
+        elif label in CONDITION["class"]["labels"]:
+            if value == "testclass":
+                score += 1000 + CONDITION["class"]["modifier"]
             else:
                 score = 0
                 break
-        elif label in CONDITION['user']['labels']:
-            if value == 'testuser':
-                score += 1000 + CONDITION['user']['modifier']
+        elif label in CONDITION["hostname"]["labels"]:
+            if value == "testhost":
+                score += 1000 + CONDITION["hostname"]["modifier"]
+            else:
+                score = 0
+                break
+        elif label in CONDITION["user"]["labels"]:
+            if value == "testuser":
+                score += 1000 + CONDITION["user"]["modifier"]
             else:
                 score = 0
                 break
@@ -80,175 +94,187 @@ def calculate_score(filename):
     return score
 
 
-@pytest.mark.parametrize(
-    'default', ['default', None], ids=['default', 'no-default'])
-@pytest.mark.parametrize(
-    'system', ['system', None], ids=['system', 'no-system'])
-@pytest.mark.parametrize(
-    'distro', ['distro', None], ids=['distro', 'no-distro'])
-@pytest.mark.parametrize(
-    'cla', ['class', None], ids=['class', 'no-class'])
-@pytest.mark.parametrize(
-    'host', ['hostname', None], ids=['hostname', 'no-host'])
-@pytest.mark.parametrize(
-    'user', ['user', None], ids=['user', 'no-user'])
-def test_score_values(
-        runner, yadm, default, system, distro, cla, host, user):
+@pytest.mark.parametrize("default", ["default", None], ids=["default", "no-default"])
+@pytest.mark.parametrize("arch", ["arch", None], ids=["arch", "no-arch"])
+@pytest.mark.parametrize("system", ["system", None], ids=["system", "no-system"])
+@pytest.mark.parametrize("distro", ["distro", None], ids=["distro", "no-distro"])
+@pytest.mark.parametrize("cla", ["class", None], ids=["class", "no-class"])
+@pytest.mark.parametrize("host", ["hostname", None], ids=["hostname", "no-host"])
+@pytest.mark.parametrize("user", ["user", None], ids=["user", "no-user"])
+def test_score_values(runner, yadm, default, arch, system, distro, cla, host, user):
     """Test score results"""
     # pylint: disable=too-many-branches
-    local_class = 'testclass'
-    local_system = 'testsystem'
-    local_distro = 'testdistro'
-    local_host = 'testhost'
-    local_user = 'testuser'
-    filenames = {'filename##': 0}
+    local_class = "testclass"
+    local_arch = "testarch"
+    local_system = "testsystem"
+    local_distro = "testdistro"
+    local_host = "testhost"
+    local_user = "testuser"
+    filenames = {"filename##": 0}
 
     if default:
         for filename in list(filenames):
-            for label in CONDITION[default]['labels']:
+            for label in CONDITION[default]["labels"]:
                 newfile = filename
-                if not newfile.endswith('##'):
-                    newfile += ','
+                if not newfile.endswith("##"):
+                    newfile += ","
                 newfile += label
                 filenames[newfile] = calculate_score(newfile)
+    if arch:
+        for filename in list(filenames):
+            for match in [True, False]:
+                for label in CONDITION[arch]["labels"]:
+                    newfile = filename
+                    if not newfile.endswith("##"):
+                        newfile += ","
+                    newfile += ".".join([label, local_arch if match else "badarch"])
+                    filenames[newfile] = calculate_score(newfile)
     if system:
         for filename in list(filenames):
             for match in [True, False]:
-                for label in CONDITION[system]['labels']:
+                for label in CONDITION[system]["labels"]:
                     newfile = filename
-                    if not newfile.endswith('##'):
-                        newfile += ','
-                    newfile += '.'.join([
-                        label,
-                        local_system if match else 'badsys'
-                        ])
+                    if not newfile.endswith("##"):
+                        newfile += ","
+                    newfile += ".".join([label, local_system if match else "badsys"])
                     filenames[newfile] = calculate_score(newfile)
     if distro:
         for filename in list(filenames):
             for match in [True, False]:
-                for label in CONDITION[distro]['labels']:
+                for label in CONDITION[distro]["labels"]:
                     newfile = filename
-                    if not newfile.endswith('##'):
-                        newfile += ','
-                    newfile += '.'.join([
-                        label,
-                        local_distro if match else 'baddistro'
-                        ])
+                    if not newfile.endswith("##"):
+                        newfile += ","
+                    newfile += ".".join([label, local_distro if match else "baddistro"])
                     filenames[newfile] = calculate_score(newfile)
     if cla:
         for filename in list(filenames):
             for match in [True, False]:
-                for label in CONDITION[cla]['labels']:
+                for label in CONDITION[cla]["labels"]:
                     newfile = filename
-                    if not newfile.endswith('##'):
-                        newfile += ','
-                    newfile += '.'.join([
-                        label,
-                        local_class if match else 'badclass'
-                        ])
+                    if not newfile.endswith("##"):
+                        newfile += ","
+                    newfile += ".".join([label, local_class if match else "badclass"])
                     filenames[newfile] = calculate_score(newfile)
     if host:
         for filename in list(filenames):
             for match in [True, False]:
-                for label in CONDITION[host]['labels']:
+                for label in CONDITION[host]["labels"]:
                     newfile = filename
-                    if not newfile.endswith('##'):
-                        newfile += ','
-                    newfile += '.'.join([
-                        label,
-                        local_host if match else 'badhost'
-                        ])
+                    if not newfile.endswith("##"):
+                        newfile += ","
+                    newfile += ".".join([label, local_host if match else "badhost"])
                     filenames[newfile] = calculate_score(newfile)
     if user:
         for filename in list(filenames):
             for match in [True, False]:
-                for label in CONDITION[user]['labels']:
+                for label in CONDITION[user]["labels"]:
                     newfile = filename
-                    if not newfile.endswith('##'):
-                        newfile += ','
-                    newfile += '.'.join([
-                        label,
-                        local_user if match else 'baduser'
-                        ])
+                    if not newfile.endswith("##"):
+                        newfile += ","
+                    newfile += ".".join([label, local_user if match else "baduser"])
                     filenames[newfile] = calculate_score(newfile)
 
     script = f"""
         YADM_TEST=1 source {yadm}
         score=0
         local_class={local_class}
+        local_classes=({local_class})
+        local_arch={local_arch}
         local_system={local_system}
         local_distro={local_distro}
         local_host={local_host}
         local_user={local_user}
     """
-    expected = ''
-    for filename in filenames:
+    expected = ""
+    for filename, score in filenames.items():
         script += f"""
             score_file "{filename}"
             echo "{filename}"
             echo "$score"
         """
-        expected += filename + '\n'
-        expected += str(filenames[filename]) + '\n'
-    run = runner(command=['bash'], inp=script)
+        expected += filename + "\n"
+        expected += str(score) + "\n"
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
+    assert run.out == expected
+
+
+@pytest.mark.parametrize("ext", [None, "e", "extension"])
+def test_extensions(runner, yadm, ext):
+    """Verify extensions do not effect scores"""
+    local_user = "testuser"
+    filename = f"filename##u.{local_user}"
+    if ext:
+        filename += f",{ext}.xyz"
+    expected = ""
+    script = f"""
+        YADM_TEST=1 source {yadm}
+        score=0
+        local_user={local_user}
+        score_file "{filename}"
+        echo "$score"
+    """
+    expected = f'{1000 + CONDITION["user"]["modifier"]}\n'
+    run = runner(command=["bash"], inp=script)
+    assert run.success
+    assert run.err == ""
     assert run.out == expected
 
 
 def test_score_values_templates(runner, yadm):
     """Test score results"""
-    local_class = 'testclass'
-    local_system = 'testsystem'
-    local_distro = 'testdistro'
-    local_host = 'testhost'
-    local_user = 'testuser'
-    filenames = {'filename##': 0}
+    local_class = "testclass"
+    local_arch = "arch"
+    local_system = "testsystem"
+    local_distro = "testdistro"
+    local_host = "testhost"
+    local_user = "testuser"
+    filenames = {"filename##": 0}
 
     for filename in list(filenames):
         for label in TEMPLATE_LABELS:
             newfile = filename
-            if not newfile.endswith('##'):
-                newfile += ','
-            newfile += '.'.join([label, 'testtemplate'])
+            if not newfile.endswith("##"):
+                newfile += ","
+            newfile += ".".join([label, "testtemplate"])
             filenames[newfile] = calculate_score(newfile)
 
     script = f"""
         YADM_TEST=1 source {yadm}
         score=0
         local_class={local_class}
+        local_arch={local_arch}
         local_system={local_system}
         local_distro={local_distro}
         local_host={local_host}
         local_user={local_user}
     """
-    expected = ''
-    for filename in filenames:
+    expected = ""
+    for filename, score in filenames.items():
         script += f"""
             score_file "{filename}"
             echo "{filename}"
             echo "$score"
         """
-        expected += filename + '\n'
-        expected += str(filenames[filename]) + '\n'
-    run = runner(command=['bash'], inp=script)
+        expected += filename + "\n"
+        expected += str(score) + "\n"
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
     assert run.out == expected
 
 
-@pytest.mark.parametrize(
-    'cmd_generated',
-    [True, False],
-    ids=['supported-template', 'unsupported-template'])
+@pytest.mark.parametrize("cmd_generated", [True, False], ids=["supported-template", "unsupported-template"])
 def test_template_recording(runner, yadm, cmd_generated):
     """Template should be recorded if choose_template_cmd outputs a command"""
 
-    mock = 'function choose_template_cmd() { return; }'
-    expected = ''
+    mock = "function choose_template_cmd() { return; }"
+    expected = ""
     if cmd_generated:
         mock = 'function choose_template_cmd() { echo "test_cmd"; }'
-        expected = 'template recorded'
+        expected = "template recorded"
 
     script = f"""
         YADM_TEST=1 source {yadm}
@@ -256,7 +282,41 @@ def test_template_recording(runner, yadm, cmd_generated):
         {mock}
         score_file "testfile##template.kind"
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
     assert run.out.rstrip() == expected
+
+
+def test_underscores_in_distro_and_family(runner, yadm):
+    """Test replacing spaces in distro / distro_family with underscores"""
+    local_distro = "test distro"
+    local_distro_family = "test family"
+    filenames = {
+        "filename##distro.test distro": 1004,
+        "filename##distro.test-distro": 0,
+        "filename##distro.test_distro": 1004,
+        "filename##distro_family.test family": 1008,
+        "filename##distro_family.test-family": 0,
+        "filename##distro_family.test_family": 1008,
+    }
+
+    script = f"""
+        YADM_TEST=1 source {yadm}
+        score=0
+        local_distro="{local_distro}"
+        local_distro_family="{local_distro_family}"
+    """
+    expected = ""
+    for filename, score in filenames.items():
+        script += f"""
+            score_file "{filename}"
+            echo "{filename}"
+            echo "$score"
+        """
+        expected += filename + "\n"
+        expected += str(score) + "\n"
+    run = runner(command=["bash"], inp=script)
+    assert run.success
+    assert run.err == ""
+    assert run.out == expected

test/test_unit_set_local_alt_values.py

@@ -4,74 +4,89 @@ import utils
 
 
 @pytest.mark.parametrize(
-    'override', [
+    "override",
+    [
         False,
-        'class',
-        'os',
-        'hostname',
-        'user',
-        ],
+        "class",
+        "arch",
+        "os",
+        "hostname",
+        "user",
+    ],
     ids=[
-        'no-override',
-        'override-class',
-        'override-os',
-        'override-hostname',
-        'override-user',
-        ]
-    )
-@pytest.mark.usefixtures('ds1_copy')
-def test_set_local_alt_values(
-        runner, yadm, paths, tst_sys, tst_host, tst_user, override):
+        "no-override",
+        "override-class",
+        "override-arch",
+        "override-os",
+        "override-hostname",
+        "override-user",
+    ],
+)
+@pytest.mark.usefixtures("ds1_copy")
+def test_set_local_alt_values(runner, yadm, paths, tst_arch, tst_sys, tst_host, tst_user, override):
     """Use issue_legacy_path_warning"""
     script = f"""
         YADM_TEST=1 source {yadm} &&
         set_operating_system &&
-        YADM_DIR={paths.yadm} configure_paths &&
+        YADM_DIR={paths.yadm} YADM_DATA={paths.data} configure_paths &&
         set_local_alt_values
         echo "class='$local_class'"
+        echo "arch='$local_arch'"
         echo "os='$local_system'"
         echo "host='$local_host'"
         echo "user='$local_user'"
     """
 
-    if override:
-        utils.set_local(paths, override, 'override')
+    if override == "class":
+        utils.set_local(paths, override, "first")
+        utils.set_local(paths, override, "override", add=True)
+    elif override:
+        utils.set_local(paths, override, "override")
 
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
 
-    if override == 'class':
+    if override == "class":
         assert "class='override'" in run.out
     else:
         assert "class=''" in run.out
 
-    if override == 'os':
+    if override == "arch":
+        assert "arch='override'" in run.out
+    else:
+        assert f"arch='{tst_arch}'" in run.out
+
+    if override == "os":
         assert "os='override'" in run.out
     else:
         assert f"os='{tst_sys}'" in run.out
 
-    if override == 'hostname':
-        assert f"host='override'" in run.out
+    if override == "hostname":
+        assert "host='override'" in run.out
     else:
         assert f"host='{tst_host}'" in run.out
 
-    if override == 'user':
-        assert f"user='override'" in run.out
+    if override == "user":
+        assert "user='override'" in run.out
     else:
         assert f"user='{tst_user}'" in run.out
 
 
-def test_distro(runner, yadm):
-    """Assert that local_distro is set"""
+def test_distro_and_family(runner, yadm):
+    """Assert that local_distro/local_distro_family are set"""
 
     script = f"""
         YADM_TEST=1 source {yadm}
+        function config() {{ echo "$1"; }}
         function query_distro() {{ echo "testdistro"; }}
+        function query_distro_family() {{ echo "testfamily"; }}
         set_local_alt_values
         echo "distro='$local_distro'"
+        echo "distro_family='$local_distro_family'"
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
-    assert run.out.strip() == "distro='testdistro'"
+    assert run.err == ""
+    assert "distro='testdistro'" in run.out
+    assert "distro_family='testfamily'" in run.out

test/test_unit_set_os.py

@@ -4,23 +4,27 @@ import pytest
 
 
 @pytest.mark.parametrize(
-    'proc_value, expected_os', [
-        ('missing', 'uname'),
-        ('has Microsoft inside', 'WSL'),
-        ('another value', 'uname'),
-    ], ids=[
-        '/proc/version missing',
-        '/proc/version includes MS',
-        '/proc/version excludes MS',
-    ])
-def test_set_operating_system(
-        runner, paths, tst_sys, proc_value, expected_os):
+    "proc_value, expected_os",
+    [
+        ("missing", "uname"),
+        ("has microsoft inside", "WSL"),  # case insensitive
+        ("has Microsoft inside", "WSL"),  # case insensitive
+        ("another value", "uname"),
+    ],
+    ids=[
+        "/proc/version missing",
+        "/proc/version includes ms",
+        "/proc/version excludes Ms",
+        "another value",
+    ],
+)
+def test_set_operating_system(runner, paths, tst_sys, proc_value, expected_os):
     """Run set_operating_system and test result"""
 
     # Normally /proc/version (set in PROC_VERSION) is inspected to identify
     # WSL. During testing, we will override that value.
-    proc_version = paths.root.join('proc_version')
-    if proc_value != 'missing':
+    proc_version = paths.root.join("proc_version")
+    if proc_value != "missing":
         proc_version.write(proc_value)
     script = f"""
         YADM_TEST=1 source {paths.pgm}
@@ -28,9 +32,9 @@ def test_set_operating_system(
         set_operating_system
         echo $OPERATING_SYSTEM
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
-    if expected_os == 'uname':
+    assert run.err == ""
+    if expected_os == "uname":
         expected_os = tst_sys
     assert run.out.rstrip() == expected_os

test/test_unit_set_yadm_dir.py

@@ -1,35 +1,43 @@
-"""Unit tests: set_yadm_dir"""
+"""Unit tests: set_yadm_dirs"""
 import pytest
 
 
 @pytest.mark.parametrize(
-    'condition',
-    ['basic', 'override', 'xdg_config_home', 'legacy'],
-    )
-def test_set_yadm_dir(runner, yadm, condition):
-    """Test set_yadm_dir"""
-    setup = ''
-    if condition == 'override':
-        setup = 'YADM_DIR=/override'
-    elif condition == 'xdg_config_home':
-        setup = 'XDG_CONFIG_HOME=/xdg'
-    elif condition == 'legacy':
-        setup = 'YADM_COMPATIBILITY=1'
+    "condition",
+    ["basic", "override", "override_data", "xdg_config_home", "xdg_data_home"],
+)
+def test_set_yadm_dirs(runner, yadm, condition):
+    """Test set_yadm_dirs"""
+    setup = ""
+    if condition == "override":
+        setup = "YADM_DIR=/override"
+    elif condition == "override_data":
+        setup = "YADM_DATA=/override"
+    elif condition == "xdg_config_home":
+        setup = "XDG_CONFIG_HOME=/xdg"
+    elif condition == "xdg_data_home":
+        setup = "XDG_DATA_HOME=/xdg"
     script = f"""
         HOME=/testhome
         YADM_TEST=1 source {yadm}
+        XDG_CONFIG_HOME=
+        XDG_DATA_HOME=
         {setup}
-        set_yadm_dir
-        echo "$YADM_DIR"
+        set_yadm_dirs
+        echo "YADM_DIR=$YADM_DIR"
+        echo "YADM_DATA=$YADM_DATA"
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
-    if condition == 'basic':
-        assert run.out.rstrip() == '/testhome/.config/yadm'
-    elif condition == 'override':
-        assert run.out.rstrip() == '/override'
-    elif condition == 'xdg_config_home':
-        assert run.out.rstrip() == '/xdg/yadm'
-    elif condition == 'legacy':
-        assert run.out.rstrip() == '/testhome/.yadm'
+    assert run.err == ""
+    if condition == "basic":
+        assert "YADM_DIR=/testhome/.config/yadm" in run.out
+        assert "YADM_DATA=/testhome/.local/share/yadm" in run.out
+    elif condition == "override":
+        assert "YADM_DIR=/override" in run.out
+    elif condition == "override_data":
+        assert "YADM_DATA=/override" in run.out
+    elif condition == "xdg_config_home":
+        assert "YADM_DIR=/xdg/yadm" in run.out
+    elif condition == "xdg_data_home":
+        assert "YADM_DATA=/xdg/yadm" in run.out

test/test_unit_template_default.py

@@ -1,18 +1,29 @@
 """Unit tests: template_default"""
 
+import os
+
+FILE_MODE = 0o754
+
 # these values are also testing the handling of bizarre characters
 LOCAL_CLASS = "default_Test+@-!^Class"
+LOCAL_CLASS2 = "default_Test+@-|^2nd_Class withSpace"
+LOCAL_ARCH = "default_Test+@-!^Arch"
 LOCAL_SYSTEM = "default_Test+@-!^System"
 LOCAL_HOST = "default_Test+@-!^Host"
 LOCAL_USER = "default_Test+@-!^User"
 LOCAL_DISTRO = "default_Test+@-!^Distro"
-TEMPLATE = f'''
+LOCAL_DISTRO_FAMILY = "default_Test+@-!^Family"
+ENV_VAR = "default_Test+@-!^Env"
+TEMPLATE = f"""
 start of template
-default class  = >{{{{yadm.class}}}}<
-default os     = >{{{{yadm.os}}}}<
-default host   = >{{{{yadm.hostname}}}}<
-default user   = >{{{{yadm.user}}}}<
-default distro = >{{{{yadm.distro}}}}<
+default class         = >{{{{yadm.class}}}}<
+default arch          = >{{{{yadm.arch}}}}<
+default os            = >{{{{yadm.os}}}}<
+default host          = >{{{{yadm.hostname}}}}<
+default user          = >{{{{yadm.user}}}}<
+default distro        = >{{{{yadm.distro}}}}<
+default distro_family = >{{{{yadm.distro_family}}}}<
+classes = >{{{{yadm.classes}}}}<
 {{% if yadm.class == "else1" %}}
 wrong else 1
 {{% else %}}
@@ -21,15 +32,30 @@ Included section from else
 {{% if yadm.class == "wrongclass1" %}}
 wrong class 1
 {{% endif %}}
+{{% if yadm.class != "wronglcass" %}}
+Included section from !=
+{{%     endif\t\t  %}}
 {{% if yadm.class == "{LOCAL_CLASS}" %}}
 Included section for class = {{{{yadm.class}}}} ({{{{yadm.class}}}} repeated)
 Multiple lines
 {{% else %}}
 Should not be included...
 {{% endif %}}
+{{% if yadm.class == "{LOCAL_CLASS2}" %}}
+Included section for second class
+{{% endif %}}
 {{% if yadm.class == "wrongclass2" %}}
 wrong class 2
 {{% endif %}}
+{{% if yadm.arch == "wrongarch1" %}}
+wrong arch 1
+{{% endif %}}
+{{% if yadm.arch == "{LOCAL_ARCH}" %}}
+Included section for arch = {{{{yadm.arch}}}} ({{{{yadm.arch}}}} repeated)
+{{% endif %}}
+{{% if yadm.arch == "wrongarch2" %}}
+wrong arch 2
+{{% endif %}}
 {{% if yadm.os == "wrongos1" %}}
 wrong os 1
 {{% endif %}}
@@ -66,60 +92,240 @@ Included section for distro = {{{{yadm.distro}}}} ({{{{yadm.distro}}}} again)
 {{% if yadm.distro == "wrongdistro2" %}}
 wrong distro 2
 {{% endif %}}
+{{% if yadm.distro_family == "wrongfamily1" %}}
+wrong family 1
+{{% endif %}}
+{{% if yadm.distro_family == "{LOCAL_DISTRO_FAMILY}" %}}
+Included section for distro_family = \
+{{{{yadm.distro_family}}}} ({{{{yadm.distro_family}}}} again)
+{{% endif %}}
+{{% if yadm.distro_family == "wrongfamily2" %}}
+wrong family 2
+{{% endif %}}
+{{% if env.VAR == "{ENV_VAR}" %}}
+Included section for env.VAR = {{{{env.VAR}}}} ({{{{env.VAR}}}} again)
+{{% endif %}}
+{{% if env.VAR == "wrongenvvar" %}}
+wrong env.VAR
+{{% endif %}}
+yadm.no_such_var="{{{{ yadm.no_such_var }}}}" and env.NO_SUCH_VAR="{{{{ env.NO_SUCH_VAR }}}}"
 end of template
-'''
-EXPECTED = f'''
+"""
+EXPECTED = f"""
 start of template
-default class  = >{LOCAL_CLASS}<
-default os     = >{LOCAL_SYSTEM}<
-default host   = >{LOCAL_HOST}<
-default user   = >{LOCAL_USER}<
-default distro = >{LOCAL_DISTRO}<
+default class         = >{LOCAL_CLASS}<
+default arch          = >{LOCAL_ARCH}<
+default os            = >{LOCAL_SYSTEM}<
+default host          = >{LOCAL_HOST}<
+default user          = >{LOCAL_USER}<
+default distro        = >{LOCAL_DISTRO}<
+default distro_family = >{LOCAL_DISTRO_FAMILY}<
+classes = >{LOCAL_CLASS2}
+{LOCAL_CLASS}<
 Included section from else
+Included section from !=
 Included section for class = {LOCAL_CLASS} ({LOCAL_CLASS} repeated)
 Multiple lines
+Included section for second class
+Included section for arch = {LOCAL_ARCH} ({LOCAL_ARCH} repeated)
 Included section for os = {LOCAL_SYSTEM} ({LOCAL_SYSTEM} repeated)
 Included section for host = {LOCAL_HOST} ({LOCAL_HOST} again)
 Included section for user = {LOCAL_USER} ({LOCAL_USER} repeated)
 Included section for distro = {LOCAL_DISTRO} ({LOCAL_DISTRO} again)
+Included section for distro_family = \
+{LOCAL_DISTRO_FAMILY} ({LOCAL_DISTRO_FAMILY} again)
+Included section for env.VAR = {ENV_VAR} ({ENV_VAR} again)
+yadm.no_such_var="" and env.NO_SUCH_VAR=""
 end of template
-'''
+"""
+
+INCLUDE_BASIC = "basic\n"
+INCLUDE_VARIABLES = """\
+included <{{ yadm.class }}> file
+
+empty line above
+"""
+INCLUDE_NESTED = "no newline at the end"
+
+TEMPLATE_INCLUDE = """\
+The first line
+{% include empty %}
+An empty file removes the line above
+{%include basic%}
+{% include "./variables.{{ yadm.os }}"  %}
+  {% include dir/nested %}
+Include basic again:
+{% include basic %}
+"""
+EXPECTED_INCLUDE = f"""\
+The first line
+An empty file removes the line above
+basic
+included <{LOCAL_CLASS}> file
+
+empty line above
+no newline at the end
+Include basic again:
+basic
+"""
+
+TEMPLATE_NESTED_IFS = """\
+{% if yadm.user == "me" %}
+    print1
+  {% if yadm.user == "me" %}
+    print2
+  {% else %}
+    no print1
+  {% endif %}
+{% else %}
+  {% if yadm.user == "me" %}
+    no print2
+  {% else %}
+    no print3
+  {% endif %}
+{% endif %}
+{% if yadm.user != "me" %}
+    no print4
+  {% if yadm.user == "me" %}
+    no print5
+  {% else %}
+    no print6
+  {% endif %}
+{% else %}
+  {% if yadm.user == "me" %}
+    print3
+  {% else %}
+    no print7
+  {% endif %}
+{% endif %}
+"""
+EXPECTED_NESTED_IFS = """\
+    print1
+    print2
+    print3
+"""
 
 
 def test_template_default(runner, yadm, tmpdir):
     """Test template_default"""
 
-    input_file = tmpdir.join('input')
+    input_file = tmpdir.join("input")
     input_file.write(TEMPLATE, ensure=True)
-    output_file = tmpdir.join('output')
+    input_file.chmod(FILE_MODE)
+    output_file = tmpdir.join("output")
+
+    # ensure overwrite works when file exists as read-only (there is some
+    # special processing when this is encountered because some environments do
+    # not properly overwrite read-only files)
+    output_file.write("existing")
+    output_file.chmod(0o400)
 
     script = f"""
         YADM_TEST=1 source {yadm}
+        set_awk
         local_class="{LOCAL_CLASS}"
+        local_classes=("{LOCAL_CLASS2}" "{LOCAL_CLASS}")
+        local_arch="{LOCAL_ARCH}"
         local_system="{LOCAL_SYSTEM}"
         local_host="{LOCAL_HOST}"
         local_user="{LOCAL_USER}"
         local_distro="{LOCAL_DISTRO}"
+        local_distro_family="{LOCAL_DISTRO_FAMILY}"
         template_default "{input_file}" "{output_file}"
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script, env={"VAR": ENV_VAR})
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
     assert output_file.read() == EXPECTED
+    assert os.stat(output_file).st_mode == os.stat(input_file).st_mode
 
 
 def test_source(runner, yadm, tmpdir):
     """Test yadm.source"""
 
-    input_file = tmpdir.join('input')
-    input_file.write('{{yadm.source}}', ensure=True)
-    output_file = tmpdir.join('output')
+    input_file = tmpdir.join("input")
+    input_file.write("{{yadm.source}}", ensure=True)
+    input_file.chmod(FILE_MODE)
+    output_file = tmpdir.join("output")
 
     script = f"""
         YADM_TEST=1 source {yadm}
+        set_awk
         template_default "{input_file}" "{output_file}"
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
     assert output_file.read().strip() == str(input_file)
+    assert os.stat(output_file).st_mode == os.stat(input_file).st_mode
+
+
+def test_include(runner, yadm, tmpdir):
+    """Test include"""
+
+    empty_file = tmpdir.join("empty")
+    empty_file.write("", ensure=True)
+
+    basic_file = tmpdir.join("basic")
+    basic_file.write(INCLUDE_BASIC)
+
+    variables_file = tmpdir.join(f"variables.{LOCAL_SYSTEM}")
+    variables_file.write(INCLUDE_VARIABLES)
+
+    nested_file = tmpdir.join("dir").join("nested")
+    nested_file.write(INCLUDE_NESTED, ensure=True)
+
+    input_file = tmpdir.join("input")
+    input_file.write(TEMPLATE_INCLUDE)
+    input_file.chmod(FILE_MODE)
+    output_file = tmpdir.join("output")
+
+    script = f"""
+        YADM_TEST=1 source {yadm}
+        set_awk
+        local_class="{LOCAL_CLASS}"
+        local_system="{LOCAL_SYSTEM}"
+        template_default "{input_file}" "{output_file}"
+    """
+    run = runner(command=["bash"], inp=script)
+    assert run.success
+    assert run.err == ""
+    assert output_file.read() == EXPECTED_INCLUDE
+    assert os.stat(output_file).st_mode == os.stat(input_file).st_mode
+
+
+def test_nested_ifs(runner, yadm, tmpdir):
+    """Test nested if statements"""
+
+    input_file = tmpdir.join("input")
+    input_file.write(TEMPLATE_NESTED_IFS, ensure=True)
+    output_file = tmpdir.join("output")
+
+    script = f"""
+        YADM_TEST=1 source {yadm}
+        set_awk
+        local_user="me"
+        template_default "{input_file}" "{output_file}"
+    """
+    run = runner(command=["bash"], inp=script)
+    assert run.success
+    assert run.err == ""
+    assert output_file.read() == EXPECTED_NESTED_IFS
+
+
+def test_env(runner, yadm, tmpdir):
+    """Test env"""
+
+    input_file = tmpdir.join("input")
+    input_file.write("{{env.PWD}}", ensure=True)
+    output_file = tmpdir.join("output")
+
+    script = f"""
+        YADM_TEST=1 source {yadm}
+        set_awk
+        template_default "{input_file}" "{output_file}"
+    """
+    run = runner(command=["bash"], inp=script)
+    assert run.success
+    assert run.err == ""
+    assert output_file.read().strip() == os.environ["PWD"]

test/test_unit_template_esh.py

@@ -0,0 +1,167 @@
+"""Unit tests: template_esh"""
+import os
+
+FILE_MODE = 0o754
+
+LOCAL_CLASS = "esh_Test+@-!^Class"
+LOCAL_CLASS2 = "esh_Test+@-|^2nd_Class withSpace"
+LOCAL_ARCH = "esh_Test+@-!^Arch"
+LOCAL_SYSTEM = "esh_Test+@-!^System"
+LOCAL_HOST = "esh_Test+@-!^Host"
+LOCAL_USER = "esh_Test+@-!^User"
+LOCAL_DISTRO = "esh_Test+@-!^Distro"
+LOCAL_DISTRO_FAMILY = "esh_Test+@-!^Family"
+TEMPLATE = f"""
+start of template
+esh class         = ><%=$YADM_CLASS%><
+esh arch          = ><%=$YADM_ARCH%><
+esh os            = ><%=$YADM_OS%><
+esh host          = ><%=$YADM_HOSTNAME%><
+esh user          = ><%=$YADM_USER%><
+esh distro        = ><%=$YADM_DISTRO%><
+esh distro_family = ><%=$YADM_DISTRO_FAMILY%><
+esh classes = ><%=$YADM_CLASSES%><
+<% if [ "$YADM_CLASS" = "wrongclass1" ]; then -%>
+wrong class 1
+<% fi -%>
+<% if [ "$YADM_CLASS" = "{LOCAL_CLASS}" ]; then -%>
+Included esh section for class = <%=$YADM_CLASS%> (<%=$YADM_CLASS%> repeated)
+<% fi -%>
+<% if [ "$YADM_CLASS" = "wrongclass2" ]; then -%>
+wrong class 2
+<% fi -%>
+<% echo "$YADM_CLASSES" | while IFS='' read cls; do
+   if [ "$cls" = "{LOCAL_CLASS2}" ]; then -%>
+Included esh section for second class
+<% fi; done -%>
+<% if [ "$YADM_ARCH" = "wrongarch1" ]; then -%>
+wrong arch 1
+<% fi -%>
+<% if [ "$YADM_ARCH" = "{LOCAL_ARCH}" ]; then -%>
+Included esh section for arch = <%=$YADM_ARCH%> (<%=$YADM_ARCH%> repeated)
+<% fi -%>
+<% if [ "$YADM_ARCH" = "wrongarch2" ]; then -%>
+wrong arch 2
+<% fi -%>
+<% if [ "$YADM_OS" = "wrongos1" ]; then -%>
+wrong os 1
+<% fi -%>
+<% if [ "$YADM_OS" = "{LOCAL_SYSTEM}" ]; then -%>
+Included esh section for os = <%=$YADM_OS%> (<%=$YADM_OS%> repeated)
+<% fi -%>
+<% if [ "$YADM_OS" = "wrongos2" ]; then -%>
+wrong os 2
+<% fi -%>
+<% if [ "$YADM_HOSTNAME" = "wronghost1" ]; then -%>
+wrong host 1
+<% fi -%>
+<% if [ "$YADM_HOSTNAME" = "{LOCAL_HOST}" ]; then -%>
+Included esh section for host = <%=$YADM_HOSTNAME%> (<%=$YADM_HOSTNAME%> again)
+<% fi -%>
+<% if [ "$YADM_HOSTNAME" = "wronghost2" ]; then -%>
+wrong host 2
+<% fi -%>
+<% if [ "$YADM_USER" = "wronguser1" ]; then -%>
+wrong user 1
+<% fi -%>
+<% if [ "$YADM_USER" = "{LOCAL_USER}" ]; then -%>
+Included esh section for user = <%=$YADM_USER%> (<%=$YADM_USER%> repeated)
+<% fi -%>
+<% if [ "$YADM_USER" = "wronguser2" ]; then -%>
+wrong user 2
+<% fi -%>
+<% if [ "$YADM_DISTRO" = "wrongdistro1" ]; then -%>
+wrong distro 1
+<% fi -%>
+<% if [ "$YADM_DISTRO" = "{LOCAL_DISTRO}" ]; then -%>
+Included esh section for distro = <%=$YADM_DISTRO%> (<%=$YADM_DISTRO%> again)
+<% fi -%>
+<% if [ "$YADM_DISTRO" = "wrongdistro2" ]; then -%>
+wrong distro 2
+<% fi -%>
+<% if [ "$YADM_DISTRO_FAMILY" = "wrongfamily1" ]; then -%>
+wrong family 1
+<% fi -%>
+<% if [ "$YADM_DISTRO_FAMILY" = "{LOCAL_DISTRO_FAMILY}" ]; then -%>
+Included esh section for distro_family = \
+<%=$YADM_DISTRO_FAMILY%> (<%=$YADM_DISTRO_FAMILY%> again)
+<% fi -%>
+<% if [ "$YADM_DISTRO" = "wrongfamily2" ]; then -%>
+wrong family 2
+<% fi -%>
+end of template
+"""
+EXPECTED = f"""
+start of template
+esh class         = >{LOCAL_CLASS}<
+esh arch          = >{LOCAL_ARCH}<
+esh os            = >{LOCAL_SYSTEM}<
+esh host          = >{LOCAL_HOST}<
+esh user          = >{LOCAL_USER}<
+esh distro        = >{LOCAL_DISTRO}<
+esh distro_family = >{LOCAL_DISTRO_FAMILY}<
+esh classes = >{LOCAL_CLASS2} {LOCAL_CLASS}<
+Included esh section for class = {LOCAL_CLASS} ({LOCAL_CLASS} repeated)
+Included esh section for second class
+Included esh section for arch = {LOCAL_ARCH} ({LOCAL_ARCH} repeated)
+Included esh section for os = {LOCAL_SYSTEM} ({LOCAL_SYSTEM} repeated)
+Included esh section for host = {LOCAL_HOST} ({LOCAL_HOST} again)
+Included esh section for user = {LOCAL_USER} ({LOCAL_USER} repeated)
+Included esh section for distro = {LOCAL_DISTRO} ({LOCAL_DISTRO} again)
+Included esh section for distro_family = \
+{LOCAL_DISTRO_FAMILY} ({LOCAL_DISTRO_FAMILY} again)
+end of template
+"""
+
+
+def test_template_esh(runner, yadm, tmpdir):
+    """Test processing by esh"""
+    # pylint: disable=duplicate-code
+
+    input_file = tmpdir.join("input")
+    input_file.write(TEMPLATE, ensure=True)
+    input_file.chmod(FILE_MODE)
+    output_file = tmpdir.join("output")
+
+    # ensure overwrite works when file exists as read-only (there is some
+    # special processing when this is encountered because some environments do
+    # not properly overwrite read-only files)
+    output_file.write("existing")
+    output_file.chmod(0o400)
+
+    script = f"""
+        YADM_TEST=1 source {yadm}
+        local_class="{LOCAL_CLASS}"
+        local_classes=("{LOCAL_CLASS2}" "{LOCAL_CLASS}")
+        local_arch="{LOCAL_ARCH}"
+        local_system="{LOCAL_SYSTEM}"
+        local_host="{LOCAL_HOST}"
+        local_user="{LOCAL_USER}"
+        local_distro="{LOCAL_DISTRO}"
+        local_distro_family="{LOCAL_DISTRO_FAMILY}"
+        template_esh "{input_file}" "{output_file}"
+    """
+    run = runner(command=["bash"], inp=script)
+    assert run.success
+    assert run.err == ""
+    assert output_file.read().strip() == str(EXPECTED).strip()
+    assert os.stat(output_file).st_mode == os.stat(input_file).st_mode
+
+
+def test_source(runner, yadm, tmpdir):
+    """Test YADM_SOURCE"""
+
+    input_file = tmpdir.join("input")
+    input_file.write("<%= $YADM_SOURCE %>", ensure=True)
+    input_file.chmod(FILE_MODE)
+    output_file = tmpdir.join("output")
+
+    script = f"""
+        YADM_TEST=1 source {yadm}
+        template_esh "{input_file}" "{output_file}"
+    """
+    run = runner(command=["bash"], inp=script)
+    assert run.success
+    assert run.err == ""
+    assert output_file.read().strip() == str(input_file)
+    assert os.stat(output_file).st_mode == os.stat(input_file).st_mode

test/test_unit_template_j2.py

@@ -1,32 +1,55 @@
 """Unit tests: template_j2cli & template_envtpl"""
+import os
+
 import pytest
 
+FILE_MODE = 0o754
+
 LOCAL_CLASS = "j2_Test+@-!^Class"
+LOCAL_CLASS2 = "j2_Test+@-|^2nd_Class withSpace"
+LOCAL_ARCH = "j2_Test+@-!^Arch"
 LOCAL_SYSTEM = "j2_Test+@-!^System"
 LOCAL_HOST = "j2_Test+@-!^Host"
 LOCAL_USER = "j2_Test+@-!^User"
 LOCAL_DISTRO = "j2_Test+@-!^Distro"
-TEMPLATE = f'''
+LOCAL_DISTRO_FAMILY = "j2_Test+@-!^Family"
+TEMPLATE = f"""
 start of template
-j2 class  = >{{{{YADM_CLASS}}}}<
-j2 os     = >{{{{YADM_OS}}}}<
-j2 host   = >{{{{YADM_HOSTNAME}}}}<
-j2 user   = >{{{{YADM_USER}}}}<
-j2 distro = >{{{{YADM_DISTRO}}}}<
+j2 class         = >{{{{YADM_CLASS}}}}<
+j2 arch          = >{{{{YADM_ARCH}}}}<
+j2 os            = >{{{{YADM_OS}}}}<
+j2 host          = >{{{{YADM_HOSTNAME}}}}<
+j2 user          = >{{{{YADM_USER}}}}<
+j2 distro        = >{{{{YADM_DISTRO}}}}<
+j2 distro_family = >{{{{YADM_DISTRO_FAMILY}}}}<
+j2 classes = >{{{{YADM_CLASSES}}}}<
 {{%- if YADM_CLASS == "wrongclass1" %}}
 wrong class 1
 {{%- endif %}}
 {{%- if YADM_CLASS == "{LOCAL_CLASS}" %}}
-Included section for class = {{{{YADM_CLASS}}}} ({{{{YADM_CLASS}}}} repeated)
+Included j2 section for class = \
+{{{{YADM_CLASS}}}} ({{{{YADM_CLASS}}}} repeated)
 {{%- endif %}}
 {{%- if YADM_CLASS == "wrongclass2" %}}
 wrong class 2
 {{%- endif %}}
+{{%- if "{LOCAL_CLASS2}" in YADM_CLASSES.split("\\n") %}}
+Included j2 section for second class
+{{%- endif %}}
+{{%- if YADM_ARCH == "wrongarch1" %}}
+wrong arch 1
+{{%- endif %}}
+{{%- if YADM_ARCH == "{LOCAL_ARCH}" %}}
+Included j2 section for arch = {{{{YADM_ARCH}}}} ({{{{YADM_ARCH}}}} repeated)
+{{%- endif %}}
+{{%- if YADM_ARCH == "wrongarch2" %}}
+wrong arch 2
+{{%- endif %}}
 {{%- if YADM_OS == "wrongos1" %}}
 wrong os 1
 {{%- endif %}}
 {{%- if YADM_OS == "{LOCAL_SYSTEM}" %}}
-Included section for os = {{{{YADM_OS}}}} ({{{{YADM_OS}}}} repeated)
+Included j2 section for os = {{{{YADM_OS}}}} ({{{{YADM_OS}}}} repeated)
 {{%- endif %}}
 {{%- if YADM_OS == "wrongos2" %}}
 wrong os 2
@@ -35,7 +58,8 @@ wrong os 2
 wrong host 1
 {{%- endif %}}
 {{%- if YADM_HOSTNAME == "{LOCAL_HOST}" %}}
-Included section for host = {{{{YADM_HOSTNAME}}}} ({{{{YADM_HOSTNAME}}}} again)
+Included j2 section for host = \
+{{{{YADM_HOSTNAME}}}} ({{{{YADM_HOSTNAME}}}} again)
 {{%- endif %}}
 {{%- if YADM_HOSTNAME == "wronghost2" %}}
 wrong host 2
@@ -44,7 +68,7 @@ wrong host 2
 wrong user 1
 {{%- endif %}}
 {{%- if YADM_USER == "{LOCAL_USER}" %}}
-Included section for user = {{{{YADM_USER}}}} ({{{{YADM_USER}}}} repeated)
+Included j2 section for user = {{{{YADM_USER}}}} ({{{{YADM_USER}}}} repeated)
 {{%- endif %}}
 {{%- if YADM_USER == "wronguser2" %}}
 wrong user 2
@@ -53,65 +77,98 @@ wrong user 2
 wrong distro 1
 {{%- endif %}}
 {{%- if YADM_DISTRO == "{LOCAL_DISTRO}" %}}
-Included section for distro = {{{{YADM_DISTRO}}}} ({{{{YADM_DISTRO}}}} again)
+Included j2 section for distro = \
+{{{{YADM_DISTRO}}}} ({{{{YADM_DISTRO}}}} again)
 {{%- endif %}}
 {{%- if YADM_DISTRO == "wrongdistro2" %}}
 wrong distro 2
 {{%- endif %}}
+{{%- if YADM_DISTRO_FAMILY == "wrongfamily1" %}}
+wrong family 1
+{{%- endif %}}
+{{%- if YADM_DISTRO_FAMILY == "{LOCAL_DISTRO_FAMILY}" %}}
+Included j2 section for distro_family = \
+{{{{YADM_DISTRO_FAMILY}}}} ({{{{YADM_DISTRO_FAMILY}}}} again)
+{{%- endif %}}
+{{%- if YADM_DISTRO_FAMILY == "wrongfamily2" %}}
+wrong family 2
+{{%- endif %}}
 end of template
-'''
-EXPECTED = f'''
+"""
+EXPECTED = f"""
 start of template
-j2 class  = >{LOCAL_CLASS}<
-j2 os     = >{LOCAL_SYSTEM}<
-j2 host   = >{LOCAL_HOST}<
-j2 user   = >{LOCAL_USER}<
-j2 distro = >{LOCAL_DISTRO}<
-Included section for class = {LOCAL_CLASS} ({LOCAL_CLASS} repeated)
-Included section for os = {LOCAL_SYSTEM} ({LOCAL_SYSTEM} repeated)
-Included section for host = {LOCAL_HOST} ({LOCAL_HOST} again)
-Included section for user = {LOCAL_USER} ({LOCAL_USER} repeated)
-Included section for distro = {LOCAL_DISTRO} ({LOCAL_DISTRO} again)
+j2 class         = >{LOCAL_CLASS}<
+j2 arch          = >{LOCAL_ARCH}<
+j2 os            = >{LOCAL_SYSTEM}<
+j2 host          = >{LOCAL_HOST}<
+j2 user          = >{LOCAL_USER}<
+j2 distro        = >{LOCAL_DISTRO}<
+j2 distro_family = >{LOCAL_DISTRO_FAMILY}<
+j2 classes = >{LOCAL_CLASS2}
+{LOCAL_CLASS}<
+Included j2 section for class = {LOCAL_CLASS} ({LOCAL_CLASS} repeated)
+Included j2 section for second class
+Included j2 section for arch = {LOCAL_ARCH} ({LOCAL_ARCH} repeated)
+Included j2 section for os = {LOCAL_SYSTEM} ({LOCAL_SYSTEM} repeated)
+Included j2 section for host = {LOCAL_HOST} ({LOCAL_HOST} again)
+Included j2 section for user = {LOCAL_USER} ({LOCAL_USER} repeated)
+Included j2 section for distro = {LOCAL_DISTRO} ({LOCAL_DISTRO} again)
+Included j2 section for distro_family = \
+{LOCAL_DISTRO_FAMILY} ({LOCAL_DISTRO_FAMILY} again)
 end of template
-'''
+"""
 
 
-@pytest.mark.parametrize('processor', ('j2cli', 'envtpl'))
+@pytest.mark.parametrize("processor", ("j2cli", "envtpl"))
 def test_template_j2(runner, yadm, tmpdir, processor):
     """Test processing by j2cli & envtpl"""
+    # pylint: disable=duplicate-code
 
-    input_file = tmpdir.join('input')
+    input_file = tmpdir.join("input")
     input_file.write(TEMPLATE, ensure=True)
-    output_file = tmpdir.join('output')
+    input_file.chmod(FILE_MODE)
+    output_file = tmpdir.join("output")
+
+    # ensure overwrite works when file exists as read-only (there is some
+    # special processing when this is encountered because some environments do
+    # not properly overwrite read-only files)
+    output_file.write("existing")
+    output_file.chmod(0o400)
 
     script = f"""
         YADM_TEST=1 source {yadm}
         local_class="{LOCAL_CLASS}"
+        local_classes=("{LOCAL_CLASS2}" "{LOCAL_CLASS}")
+        local_arch="{LOCAL_ARCH}"
         local_system="{LOCAL_SYSTEM}"
         local_host="{LOCAL_HOST}"
         local_user="{LOCAL_USER}"
         local_distro="{LOCAL_DISTRO}"
+        local_distro_family="{LOCAL_DISTRO_FAMILY}"
         template_{processor} "{input_file}" "{output_file}"
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
     assert output_file.read() == EXPECTED
+    assert os.stat(output_file).st_mode == os.stat(input_file).st_mode
 
 
-@pytest.mark.parametrize('processor', ('j2cli', 'envtpl'))
+@pytest.mark.parametrize("processor", ("j2cli", "envtpl"))
 def test_source(runner, yadm, tmpdir, processor):
     """Test YADM_SOURCE"""
 
-    input_file = tmpdir.join('input')
-    input_file.write('{{YADM_SOURCE}}', ensure=True)
-    output_file = tmpdir.join('output')
+    input_file = tmpdir.join("input")
+    input_file.write("{{YADM_SOURCE}}", ensure=True)
+    input_file.chmod(FILE_MODE)
+    output_file = tmpdir.join("output")
 
     script = f"""
         YADM_TEST=1 source {yadm}
         template_{processor} "{input_file}" "{output_file}"
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
+    assert run.err == ""
     assert output_file.read().strip() == str(input_file)
+    assert os.stat(output_file).st_mode == os.stat(input_file).st_mode

test/test_unit_upgrade.py

@@ -1,57 +1,42 @@
 """Unit tests: upgrade"""
 import pytest
 
-LEGACY_PATHS = [
-    'config',
-    'encrypt',
-    'files.gpg',
-    'bootstrap',
-    'hooks/pre_command',
-    'hooks/post_command',
-]
 
-# used:
-# YADM_COMPATIBILITY
-# YADM_DIR
-# YADM_LEGACY_DIR
-# GIT_PROGRAM
-@pytest.mark.parametrize('condition', ['compat', 'equal', 'existing_repo'])
+@pytest.mark.parametrize("condition", ["override", "equal", "existing_repo"])
 def test_upgrade_errors(tmpdir, runner, yadm, condition):
     """Test upgrade() error conditions"""
 
-    compatibility = 'YADM_COMPATIBILITY=1' if condition == 'compat' else ''
-
-    home = tmpdir.mkdir('home')
-    yadm_dir = home.join('.config/yadm')
-    legacy_dir = home.join('.yadm')
-    if condition == 'equal':
-        legacy_dir = yadm_dir
-    if condition == 'existing_repo':
-        yadm_dir.ensure_dir('repo.git')
-        legacy_dir.ensure_dir('repo.git')
+    home = tmpdir.mkdir("home")
+    yadm_dir = home.join(".config/yadm")
+    yadm_data = home.join(".local/share/yadm")
+    override = ""
+    if condition == "override":
+        override = "override"
+    if condition == "equal":
+        yadm_data = yadm_dir
+    if condition == "existing_repo":
+        yadm_dir.ensure_dir("repo.git")
+        yadm_data.ensure_dir("repo.git")
 
     script = f"""
         YADM_TEST=1 source {yadm}
-        {compatibility}
         YADM_DIR="{yadm_dir}"
-        YADM_REPO="{yadm_dir}/repo.git"
-        YADM_LEGACY_DIR="{legacy_dir}"
+        YADM_DATA="{yadm_data}"
+        YADM_REPO="{yadm_data}/repo.git"
+        YADM_LEGACY_ARCHIVE="files.gpg"
+        YADM_OVERRIDE_REPO="{override}"
         upgrade
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.failure
-    assert run.err == ''
-    assert 'Unable to upgrade' in run.out
-    if condition == 'compat':
-        assert 'YADM_COMPATIBILITY' in run.out
-    if condition == 'equal':
-        assert 'has been resolved as' in run.out
-    if condition == 'existing_repo':
-        assert 'already exists' in run.out
+    assert "Unable to upgrade" in run.err
+    if condition in ["override", "equal"]:
+        assert "Paths have been overridden" in run.err
+    elif condition == "existing_repo":
+        assert "already exists" in run.err
 
 
-@pytest.mark.parametrize(
-    'condition', ['no-paths', 'untracked', 'tracked', 'submodules'])
+@pytest.mark.parametrize("condition", ["no-paths", "untracked", "tracked", "submodules"])
 def test_upgrade(tmpdir, runner, yadm, condition):
     """Test upgrade()
 
@@ -59,62 +44,70 @@ def test_upgrade(tmpdir, runner, yadm, condition):
     mock for git. echo will return true, simulating a positive result from "git
     ls-files". Also echo will report the parameters for "git mv".
     """
-    home = tmpdir.mkdir('home')
-    yadm_dir = home.join('.config/yadm')
-    legacy_dir = home.join('.yadm')
+    legacy_paths = ("config", "encrypt", "bootstrap", "hooks/pre_cmd")
+    home = tmpdir.mkdir("home")
+    yadm_dir = home.join(".config/yadm")
+    yadm_data = home.join(".local/share/yadm")
+    yadm_legacy = home.join(".yadm")
 
-    if condition != 'no-paths':
-        legacy_dir.join('repo.git/config').write('test-repo', ensure=True)
-        for lpath in LEGACY_PATHS:
-            legacy_dir.join(lpath).write(lpath, ensure=True)
+    if condition != "no-paths":
+        yadm_dir.join("repo.git/config").write("test-repo", ensure=True)
+        yadm_dir.join("files.gpg").write("files.gpg", ensure=True)
+        for path in legacy_paths:
+            yadm_legacy.join(path).write(path, ensure=True)
 
     mock_git = ""
-    if condition in ['tracked', 'submodules']:
-        mock_git = f'''
+    if condition != "no-paths":
+        mock_git = f"""
             function git() {{
                 echo "$@"
-                if [[ "$*" == *.gitmodules* ]]; then
-                    return { '0' if condition == 'submodules' else '1' }
+                if [[ "$*" = *"submodule status" ]]; then
+                    { 'echo " 1234567 mymodule (1.0)"'
+                        if condition == 'submodules' else ':' }
+                fi
+                if [[ "$*" = *ls-files* ]]; then
+                    return { 1 if condition == 'untracked' else 0 }
                 fi
                 return 0
             }}
-        '''
+        """
 
     script = f"""
         YADM_TEST=1 source {yadm}
+        YADM_LEGACY_DIR="{yadm_legacy}"
         YADM_DIR="{yadm_dir}"
-        YADM_REPO="{yadm_dir}/repo.git"
-        YADM_LEGACY_DIR="{legacy_dir}"
+        YADM_DATA="{yadm_data}"
+        YADM_REPO="{yadm_data}/repo.git"
+        YADM_ARCHIVE="{yadm_data}/archive"
         GIT_PROGRAM="git"
         {mock_git}
         function cd {{ echo "$@";}}
         upgrade
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success
-    assert run.err == ''
-    if condition == 'no-paths':
-        assert 'Upgrade is not necessary' in run.out
+    assert run.err == ""
+    if condition == "no-paths":
+        assert "Upgrade is not necessary" in run.out
     else:
-        for lpath in LEGACY_PATHS + ['repo.git']:
-            expected = (
-                f'Moving {legacy_dir.join(lpath)} '
-                f'to {yadm_dir.join(lpath)}')
+        for lpath, npath in [("repo.git", "repo.git"), ("files.gpg", "archive")]:
+            expected = f"Moving {yadm_dir.join(lpath)} " f"to {yadm_data.join(npath)}"
             assert expected in run.out
-        if condition == 'untracked':
-            assert 'test-repo' in yadm_dir.join('repo.git/config').read()
-            for lpath in LEGACY_PATHS:
-                assert lpath in yadm_dir.join(lpath).read()
-        elif condition in ['tracked', 'submodules']:
-            for lpath in LEGACY_PATHS:
-                expected = (
-                    f'mv {legacy_dir.join(lpath)} '
-                    f'{yadm_dir.join(lpath)}')
-                assert expected in run.out
-            assert 'files tracked by yadm have been renamed' in run.out
-            if condition == 'submodules':
-                assert 'submodule deinit -f .' in run.out
-                assert 'submodule update --init --recursive' in run.out
+        for path in legacy_paths:
+            expected = f"Moving {yadm_legacy.join(path)} " f"to {yadm_dir.join(path)}"
+            assert expected in run.out
+        if condition == "untracked":
+            assert "test-repo" in yadm_data.join("repo.git/config").read()
+            assert "files.gpg" in yadm_data.join("archive").read()
+            for path in legacy_paths:
+                assert path in yadm_dir.join(path).read()
+        elif condition in ["tracked", "submodules"]:
+            expected = f'mv {yadm_dir.join("files.gpg")} ' f'{yadm_data.join("archive")}'
+            assert expected in run.out
+            assert "files tracked by yadm have been renamed" in run.out
+            if condition == "submodules":
+                assert "submodule deinit -- mymodule" in run.out
+                assert "submodule update --init --recursive -- mymodule" in run.out
             else:
-                assert 'submodule deinit -f .' not in run.out
-                assert 'submodule update --init --recursive' not in run.out
+                assert "submodule deinit" not in run.out
+                assert "submodule update --init --recursive" not in run.out

test/test_unit_x_program.py

@@ -1,46 +1,50 @@
 """Unit tests: yadm.[git,gpg]-program"""
 
 import os
+
 import pytest
 
 
 @pytest.mark.parametrize(
-    'executable, success, value, match', [
-        (None, True, 'program', None),
-        ('cat', True, 'cat', None),
-        ('badprogram', False, None, 'badprogram'),
-    ], ids=[
-        'executable missing',
-        'valid alternative',
-        'invalid alternative',
-    ])
-@pytest.mark.parametrize('program', ['git', 'gpg'])
-def test_x_program(
-        runner, yadm_y, paths, program, executable, success, value, match):
+    "executable, success, value, match",
+    [
+        (None, True, "program", None),
+        ("cat", True, "cat", None),
+        ("badprogram", False, None, "badprogram"),
+    ],
+    ids=[
+        "executable missing",
+        "valid alternative",
+        "invalid alternative",
+    ],
+)
+@pytest.mark.parametrize("program", ["git", "gpg"])
+def test_x_program(runner, yadm_cmd, paths, program, executable, success, value, match):
     """Set yadm.X-program, and test result of require_X"""
 
     # set configuration
     if executable:
-        os.system(' '.join(yadm_y(
-            'config', f'yadm.{program}-program', executable)))
+        os.system(" ".join(yadm_cmd("config", f"yadm.{program}-program", executable)))
 
     # test require_[git,gpg]
     script = f"""
         YADM_TEST=1 source {paths.pgm}
-        YADM_CONFIG="{paths.config}"
+        YADM_OVERRIDE_CONFIG="{paths.config}"
+        configure_paths
         require_{program}
         echo ${program.upper()}_PROGRAM
     """
-    run = runner(command=['bash'], inp=script)
+    run = runner(command=["bash"], inp=script)
     assert run.success == success
-    assert run.err == ''
 
     # [GIT,GPG]_PROGRAM set correctly
-    if value == 'program':
+    if value == "program":
         assert run.out.rstrip() == program
     elif value:
         assert run.out.rstrip() == value
 
     # error reported about bad config
     if match:
-        assert match in run.out
+        assert match in run.err
+    else:
+        assert run.err == ""

test/test_upgrade.py

@@ -0,0 +1,131 @@
+"""Test upgrade"""
+
+import os
+
+import pytest
+
+
+@pytest.mark.parametrize(
+    "versions",
+    [
+        ("1.12.0", "2.5.0"),
+        ("1.12.0",),
+        ("2.5.0",),
+    ],
+    ids=[
+        "1.12.0 -> 2.5.0 -> latest",
+        "1.12.0 -> latest",
+        "2.5.0 -> latest",
+    ],
+)
+@pytest.mark.parametrize("submodule", [False, True], ids=["no submodule", "with submodules"])
+def test_upgrade(tmpdir, runner, versions, submodule):
+    """Upgrade tests"""
+    # pylint: disable=too-many-statements
+    home = tmpdir.mkdir("HOME")
+    env = {"HOME": str(home)}
+    runner(["git", "config", "--global", "init.defaultBranch", "master"], env=env)
+    runner(["git", "config", "--global", "protocol.file.allow", "always"], env=env)
+
+    if submodule:
+        ext_repo = tmpdir.mkdir("ext_repo")
+        ext_repo.join("afile").write("some data")
+
+        for cmd in (("init",), ("add", "afile"), ("commit", "-m", "test")):
+            run = runner(["git", "-C", str(ext_repo), *cmd])
+            assert run.success
+
+    os.environ.pop("XDG_CONFIG_HOME", None)
+    os.environ.pop("XDG_DATA_HOME", None)
+
+    def run_version(version, *args, check_stderr=True):
+        yadm = f"yadm-{version}" if version else "/yadm/yadm"
+        run = runner([yadm, *args], shell=True, cwd=str(home), env=env)
+        assert run.success
+        if check_stderr:
+            assert run.err == ""
+        return run
+
+    # Initialize the repo with the first version
+    first = versions[0]
+    run_version(first, "init")
+
+    home.join("file").write("some data")
+    run_version(first, "add", "file")
+    run_version(first, "commit", "-m", '"First commit"')
+
+    if submodule:
+        # When upgrading via 2.5.0 we can't have a submodule that's been added
+        # after being cloned as 2.5.0 fails the upgrade in that case.
+        can_upgrade_cloned_submodule = "2.5.0" not in versions[1:]
+        if can_upgrade_cloned_submodule:
+            # Check out a repo and then add it as a submodule
+            run = runner(["git", "-C", str(home), "clone", str(ext_repo), "b"])
+            assert run.success
+            run_version(first, "submodule", "add", str(ext_repo), "b")
+
+        # Add submodule without first checking it out
+        run_version(first, "submodule", "add", str(ext_repo), "a", check_stderr=False)
+        run_version(first, "submodule", "add", str(ext_repo), "c", check_stderr=False)
+
+        run_version(first, "commit", "-m", '"Add submodules"')
+
+    for path in (".yadm", ".config/yadm"):
+        yadm_dir = home.join(path)
+        if yadm_dir.exists():
+            break
+
+    yadm_dir.join("bootstrap").write("init stuff")
+    run_version(first, "add", yadm_dir.join("bootstrap"))
+    run_version(first, "commit", "-m", "bootstrap")
+
+    yadm_dir.join("encrypt").write("secret")
+
+    hooks_dir = yadm_dir.mkdir("hooks")
+    hooks_dir.join("pre_status").write("status")
+    hooks_dir.join("post_commit").write("commit")
+
+    run_version(first, "config", "local.class", "test")
+    run_version(first, "config", "foo.bar", "true")
+
+    # Run upgrade with intermediate versions and latest
+    latest = None
+    for version in versions[1:] + (latest,):
+        run = run_version(version, "upgrade", check_stderr=not submodule)
+        if submodule:
+            lines = run.err.splitlines()
+            if can_upgrade_cloned_submodule:
+                assert "Migrating git directory of" in lines[0]
+                assert str(home.join("b/.git")) in lines[1]
+                assert str(yadm_dir.join("repo.git/modules/b")) in lines[2]
+                del lines[:3]
+            for line in lines:
+                assert line.startswith("Submodule")
+                assert "registered for path" in line
+
+    # Verify result for the final upgrade
+    run_version(latest, "status")
+
+    run = run_version(latest, "show", "HEAD:file")
+    assert run.out == "some data"
+
+    if submodule:
+        if can_upgrade_cloned_submodule:
+            assert home.join("b/afile").read() == "some data"
+        assert home.join("a/afile").read() == "some data"
+        assert home.join("c/afile").read() == "some data"
+
+    yadm_dir = home.join(".config/yadm")
+
+    assert yadm_dir.join("bootstrap").read() == "init stuff"
+    assert yadm_dir.join("encrypt").read() == "secret"
+
+    hooks_dir = yadm_dir.join("hooks")
+    assert hooks_dir.join("pre_status").read() == "status"
+    assert hooks_dir.join("post_commit").read() == "commit"
+
+    run = run_version(latest, "config", "local.class")
+    assert run.out.rstrip() == "test"
+
+    run = run_version(latest, "config", "foo.bar")
+    assert run.out.rstrip() == "true"

test/test_version.py

@@ -1,35 +1,36 @@
 """Test version"""
 
 import re
+
 import pytest
 
 
-@pytest.fixture(scope='module')
+@pytest.fixture(scope="module")
 def expected_version(yadm):
     """
     Expected semantic version number. This is taken directly out of yadm,
     searching for the VERSION= string.
     """
-    yadm_version = re.findall(
-        r'VERSION=([^\n]+)',
-        open(yadm).read())
+    with open(yadm, encoding="utf-8") as source_file:
+        yadm_version = re.findall(r"VERSION=([^\n]+)", source_file.read())
     if yadm_version:
         return yadm_version[0]
-    pytest.fail(f'version not found in {yadm}')
-    return 'not found'
+    pytest.fail(f"version not found in {yadm}")
+    return "not found"
 
 
 def test_semantic_version(expected_version):
     """Version is semantic"""
     # semantic version conforms to MAJOR.MINOR.PATCH
-    assert re.search(r'^\d+\.\d+\.\d+$', expected_version), (
-        'does not conform to MAJOR.MINOR.PATCH')
+    assert re.search(r"^\d+\.\d+\.\d+$", expected_version), "does not conform to MAJOR.MINOR.PATCH"
 
 
-def test_reported_version(
-        runner, yadm_y, expected_version):
-    """Report correct version"""
-    run = runner(command=yadm_y('version'))
+@pytest.mark.parametrize("cmd", ["--version", "version"])
+def test_reported_version(runner, yadm_cmd, cmd, expected_version):
+    """Report correct version and bash/git versions"""
+    run = runner(command=yadm_cmd(cmd))
     assert run.success
-    assert run.err == ''
-    assert run.out == f'yadm {expected_version}\n'
+    assert run.err == ""
+    assert "bash version" in run.out
+    assert "git version" in run.out
+    assert run.out.endswith(f"\nyadm version {expected_version}\n")

test/utils.py

@@ -3,37 +3,42 @@
 This module holds values/functions common to multiple tests.
 """
 
-import re
 import os
+import re
 
-ALT_FILE1 = 'test_alt'
-ALT_FILE2 = 'test alt/test alt'
-ALT_DIR = 'test alt/test alt dir'
+ALT_FILE1 = "test_alt"
+ALT_FILE2 = "test alt/test alt"
+ALT_DIR = "test alt/test alt dir"
 
 # Directory based alternates must have a tracked contained file.
 # This will be the test contained file name
-CONTAINED = 'contained_file'
+CONTAINED = "contained_file"
 
 # These variables are used for making include files which will be processed
 # within jinja templates
-INCLUDE_FILE = 'inc_file'
-INCLUDE_DIRS = ['', 'test alt']
-INCLUDE_CONTENT = '8780846c02e34c930d0afd127906668f'
+INCLUDE_FILE = "inc_file"
+INCLUDE_DIRS = ["", "test alt"]
+INCLUDE_CONTENT = "8780846c02e34c930d0afd127906668f"
 
 
-def set_local(paths, variable, value):
+def set_local(paths, variable, value, add=False):
     """Set local override"""
-    os.system(
-        f'GIT_DIR={str(paths.repo)} '
-        f'git config --local "local.{variable}" "{value}"'
-    )
+    add = "--add" if add else ""
+    os.system(f"GIT_DIR={str(paths.repo)} " f'git config --local {add} "local.{variable}" "{value}"')
 
 
-def create_alt_files(paths, suffix,
-                     preserve=False, tracked=True,
-                     encrypt=False, exclude=False,
-                     content=None, includefile=False,
-                     yadm_alt=False, yadm_dir=None):
+def create_alt_files(
+    paths,
+    suffix,
+    preserve=False,
+    tracked=True,
+    encrypt=False,
+    exclude=False,
+    content=None,
+    includefile=False,
+    yadm_alt=False,
+    yadm_dir=None,
+):
     """Create new files, and add to the repo
 
     This is used for testing alternate files. In each case, a suffix is
@@ -41,7 +46,7 @@ def create_alt_files(paths, suffix,
     repo handling are dependent upon the function arguments.
     """
 
-    basepath = yadm_dir.join('alt') if yadm_alt else paths.work
+    basepath = yadm_dir.join("alt") if yadm_alt else paths.work
 
     if not preserve:
         for remove_path in (ALT_FILE1, ALT_FILE2, ALT_DIR):
@@ -59,28 +64,28 @@ def create_alt_files(paths, suffix,
     # Do not test directory support for jinja alternates
     test_paths = [new_file1, new_file2]
     test_names = [ALT_FILE1, ALT_FILE2]
-    if not re.match(r'##(t$|t\.|template|yadm)', suffix):
+    if not re.match(r"##(t$|t\.|template|yadm)", suffix):
         test_paths += [new_dir]
         test_names += [ALT_DIR]
 
     for test_path in test_paths:
         if content:
-            test_path.write('\n' + content, mode='a', ensure=True)
+            test_path.write("\n" + content, mode="a", ensure=True)
         assert test_path.exists()
 
     _create_includefiles(includefile, test_paths, basepath)
     _create_tracked(tracked, test_paths, paths)
 
-    prefix = '.config/yadm/alt/' if yadm_alt else ''
+    prefix = ".config/yadm/alt/" if yadm_alt else ""
     _create_encrypt(encrypt, test_names, suffix, paths, exclude, prefix)
 
 
 def parse_alt_output(output, linked=True):
     """Parse output of 'alt', and return list of linked files"""
-    regex = r'Creating (.+) from template (.+)$'
+    regex = r"Creating (.+) from template (.+)$"
     if linked:
-        regex = r'Linking (.+) to (.+)$'
-    parsed_list = dict()
+        regex = r"Linking (.+) to (.+)$"
+    parsed_list = {}
     for line in output.splitlines():
         match = re.match(regex, line)
         if match:
@@ -94,7 +99,7 @@ def parse_alt_output(output, linked=True):
 def _create_includefiles(includefile, test_paths, basepath):
     if includefile:
         for dpath in INCLUDE_DIRS:
-            incfile = basepath.join(dpath + '/' + INCLUDE_FILE)
+            incfile = basepath.join(dpath + "/" + INCLUDE_FILE)
             incfile.write(INCLUDE_CONTENT, ensure=True)
             test_paths += [incfile]
 
@@ -109,8 +114,6 @@ def _create_tracked(tracked, test_paths, paths):
 def _create_encrypt(encrypt, test_names, suffix, paths, exclude, prefix):
     if encrypt:
         for encrypt_name in test_names:
-            paths.encrypt.write(
-                f'{prefix + encrypt_name + suffix}\n', mode='a')
+            paths.encrypt.write(f"{prefix + encrypt_name + suffix}\n", mode="a")
             if exclude:
-                paths.encrypt.write(
-                    f'!{prefix + encrypt_name + suffix}\n', mode='a')
+                paths.encrypt.write(f"!{prefix + encrypt_name + suffix}\n", mode="a")

yadm.1

@@ -1,5 +1,5 @@
-." vim: set spell so=8:
-.TH yadm 1 "12 November 2019" "2.0.1"
+.\" vim: set spell so=8:
+.TH yadm 1 "8 November 2024" "3.3.0"
 
 .SH NAME
 
@@ -47,8 +47,6 @@ list
 
 .BR yadm " encrypt
 
-.BR yadm " enter
-
 .BR yadm " decrypt
 .RB [ -l ]
 
@@ -56,7 +54,14 @@ list
 
 .BR yadm " perms
 
+.BR yadm " enter [ command ]
+
+.BR yadm " git-crypt [ options ]
+
+.BR yadm " transcrypt [ options ]
+
 .BR yadm " upgrade
+.RB [ -f ]
 
 .BR yadm " introspect
 .I category
@@ -111,37 +116,15 @@ if it exists.
 .TP
 .BI clone " url
 Clone a remote repository for tracking dotfiles.
-After the contents of the remote repository have been fetched, a "merge" of
-.I origin/master
-is attempted.
+After the contents of the remote repository have been fetched, a "check out" of
+the remote HEAD branch is attempted.
 If there are conflicting files already present in the
 .IR work-tree ,
-this merge will fail and instead a "reset" of
-.I origin/master
-will be done, followed by a "stash". This "stash" operation will preserve the
-original data.
-
-You can review the stashed conflicts by running the command
-
-.RS
-.RS
-yadm stash show -p
-.RE
-
-from within your
-.I $HOME
-directory. If you want to restore the stashed data, you can run
-
-.RS
-yadm stash apply
-.RE
-or
-.RS
-yadm stash pop
-.RE
+the local version will be left unmodified and you'll have to review and resolve
+the difference.
 
 The repository is stored in
-.IR $HOME/.config/yadm/repo.git .
+.IR $HOME/.local/share/yadm/repo.git .
 By default,
 .I $HOME
 will be used as the
@@ -150,8 +133,7 @@ but this can be overridden with the
 .BR -w " option.
 yadm can be forced to overwrite an existing repository by providing the
 .BR -f " option.
-If you want to use a branch other than
-.IR origin/master ,
+If you want to use a branch other than the remote HEAD branch
 you can specify it using the
 .BR -b " option.
 By default yadm will ask the user if the bootstrap program should be run (if it
@@ -159,18 +141,17 @@ exists). The options
 .BR --bootstrap " or " --no-bootstrap
 will either force the bootstrap to be run, or prevent it from being run,
 without prompting the user.
-.RE
 .TP
 .B config
 This command manages configurations for yadm.
-This command works exactly they way
+This command works exactly the way
 .BR git-config (1)
 does.
 See the CONFIGURATION section for more details.
 .TP
 .B decrypt
 Decrypt all files stored in
-.IR $HOME/.config/yadm/files.gpg .
+.IR $HOME/.local/share/yadm/archive .
 Files decrypted will be relative to the configured
 .IR work-tree " (usually
 .IR $HOME ).
@@ -187,16 +168,37 @@ See the ENCRYPTION section for more details.
 Run a sub-shell with all Git variables set. Exit the sub-shell the same way you
 leave your normal shell (usually with the "exit" command). This sub-shell can
 be used to easily interact with your yadm repository using "git" commands. This
-could be useful if you are using a tool which uses Git directly. For example,
+could be useful if you are using a tool which uses Git directly, such as tig,
+vim-fugitive, git-cola, etc.
+
+Optionally, you can provide a command after "enter", and instead of invoking
+your shell, that command will be run with all of the Git variables exposed to
+the command's environment.
+
 Emacs Tramp and Magit can manage files by using this configuration:
+
 .RS
     (add-to-list 'tramp-methods
          '("yadm"
            (tramp-login-program "yadm")
            (tramp-login-args (("enter")))
+           (tramp-login-env (("SHELL") ("/bin/sh")))
            (tramp-remote-shell "/bin/sh")
            (tramp-remote-shell-args ("-c"))))
 .RE
+
+.RS
+With this config, use (magit-status "/yadm::").
+.RE
+.TP
+.BI git-crypt " options
+If git-crypt is installed, this command allows you to pass options directly to
+git-crypt, with the environment configured to use the yadm repository.
+
+git-crypt enables transparent encryption and decryption of files in a git repository.
+You can read
+https://github.com/AGWA/git-crypt
+for details.
 .TP
 .B gitconfig
 Pass options to the
@@ -224,7 +226,7 @@ Print a summary of yadm commands.
 .B init
 Initialize a new, empty repository for tracking dotfiles.
 The repository is stored in
-.IR $HOME/.config/yadm/repo.git .
+.IR $HOME/.local/share/yadm/repo.git .
 By default,
 .I $HOME
 will be used as the
@@ -257,50 +259,43 @@ configuration
 .I yadm.auto-perms
 to "false".
 .TP
+.BI transcrypt " options
+If transcrypt is installed, this command allows you to pass options directly to
+transcrypt, with the environment configured to use the yadm repository.
+
+transcrypt enables transparent encryption and decryption of files in a git repository.
+You can read
+https://github.com/elasticdog/transcrypt
+for details.
+.TP
 .B upgrade
-Version 2 of yadm uses a different directory for storing your configurations.
-When you start to use version 2 for the first time, you may see warnings about
+Version 3 of yadm uses a different directory for storing data.
+When you start to use version 3 for the first time, you may see warnings about
 moving your data to this new directory.
 The easiest way to accomplish this is by running "yadm upgrade".
 This command will start by moving your yadm repo to the new path.
-Next it will move any configuration data to the new path.
-If the configurations are tracked within your yadm repo, this command will
-"stage" the renaming of those files in the repo's index.
-Upgrading will also re-initialize all submodules you have added (otherwise they
-will be broken when the repo moves).
+Next it will move any archive data.
+If the archive is tracked within your yadm repo, this command will
+"stage" the renaming of that file in the repo's index.
+
+Upgrading will attempt to de-initialize and re-initialize your submodules. If
+your submodules cannot be de-initialized, the upgrade will fail. The most
+common reason submodules will fail to de-initialize is because they have local
+modifications. If you are willing to lose the local modifications to those
+submodules, you can use the
+.B -f
+option with the "upgrade" command to force the de-initialization.
+
 After running "yadm upgrade", you should run "yadm status" to review changes
 which have been staged, and commit them to your repository.
 
 You can read
-https://yadm.io/docs/upgrade_from_1
+https://yadm.io/docs/upgrade_from_2
 for more information.
 .TP
 .B version
 Print the version of yadm.
 
-.SH COMPATIBILITY
-
-Beginning with version 2.0.0, yadm introduced a couple major changes which may
-require you to adjust your configurations.
-See the
-.B upgrade
-command for help making those adjustments.
-
-First, yadm now uses the "XDG Base Directory Specification" to find its
-configurations. You can read
-https://yadm.io/docs/upgrade_from_1
-for more information.
-
-Second, the naming conventions for alternate files have been changed.
-You can read https://yadm.io/docs/alternates for more information.
-
-If you want to retain the old functionality, you can set an environment variable,
-.IR YADM_COMPATIBILITY=1 .
-Doing so will automatically use the old yadm directory, and process alternates
-the same as the pre-2.0.0 version. This compatibility mode is deprecated, and
-will be removed in future versions. This mode exists solely for transitioning
-to the new paths and naming of alternates.
-
 .SH OPTIONS
 
 yadm supports a set of universal options that alter the paths it uses. The
@@ -315,10 +310,14 @@ alias yadm='yadm --yadm-repo /alternate/path/to/repo'
 .RE
 
 The following is the full list of universal options.
-Each option should be followed by a fully qualified path.
+Each option should be followed by a path.
 .TP
 .B -Y,--yadm-dir
 Override the yadm directory.
+yadm stores its configurations relative to this directory.
+.TP
+.B --yadm-data
+Override the yadm data directory.
 yadm stores its data relative to this directory.
 .TP
 .B --yadm-repo
@@ -359,12 +358,6 @@ The following is the full list of supported configurations:
 If set to "true", alternate files will be copies instead of symbolic links.
 This might be desirable, because some systems may not properly support
 symlinks.
-
-NOTE: The deprecated
-.I yadm.cygwin-copy
-option used by older versions of yadm has been replaced by
-.IR yadm.alt-copy .
-The old option will be removed in the next version of yadm.
 .TP
 .B yadm.auto-alt
 Disable the automatic linking described in the section ALTERNATES. If disabled,
@@ -386,6 +379,11 @@ This feature is enabled by default.
 .B yadm.auto-private-dirs
 Disable the automatic creating of private directories described in the section PERMISSIONS.
 .TP
+.B yadm.cipher
+Configure which encryption system is used by the encrypt/decrypt commands.
+Valid options are "gpg" and "openssl". The default is "gpg".
+Detailed information can be found in the section ENCRYPTION.
+.TP
 .B yadm.git-program
 Specify an alternate program to use instead of "git".
 By default, the first "git" found in $PATH is used.
@@ -403,18 +401,33 @@ By default, the first "gpg" found in $PATH is used.
 Asymmetrically encrypt files with a gpg public/private key pair.
 Provide a "key ID" to specify which public key to encrypt with.
 The key must exist in your public keyrings.
+Multiple recipients can be specified (separated by space).
 If left blank or not provided, symmetric encryption is used instead.
 If set to "ASK", gpg will interactively ask for recipients.
 See the ENCRYPTION section for more details.
 This feature is disabled by default.
 .TP
+.B yadm.openssl-ciphername
+Specify which cipher should be used by openssl.
+"aes-256-cbc" is used by default.
+.TP
+.B yadm.openssl-old
+Newer versions of openssl support the pbkdf2 key derivation function. This is
+used by default. If this configuration is set to "true", openssl operations
+will use options compatible with older versions of openssl. If you change this
+option, you will need to recreate your encrypted archive.
+.TP
+.B yadm.openssl-program
+Specify an alternate program to use instead of "openssl".
+By default, the first "openssl" found in $PATH is used.
+.TP
 .B yadm.ssh-perms
 Disable the permission changes to
 .IR $HOME/.ssh/* .
 This feature is enabled by default.
 
 .RE
-The following four "local" configurations are not stored in the
+The following five "local" configurations are not stored in the
 .IR $HOME/.config/yadm/config,
 they are stored in the local repository.
 
@@ -422,6 +435,14 @@ they are stored in the local repository.
 .B local.class
 Specify a class for the purpose of symlinking alternate files.
 By default, no class will be matched.
+The local host can be assigned multiple classes using command:
+
+.RS
+yadm config --add local.class <additional-class>
+.RE
+.TP
+.B local.arch
+Override the architecture for the purpose of symlinking alternate files.
 .TP
 .B local.hostname
 Override the hostname for the purpose of symlinking alternate files.
@@ -465,15 +486,11 @@ Valid if the value matches the current user.
 Current user is calculated by running
 .BR "id -u -n" .
 .TP
-.BR distro , " d
-Valid if the value matches the distro.
-Distro is calculated by running
-.BR "lsb_release -si" .
-.TP
-.BR os , " o
-Valid if the value matches the OS.
-OS is calculated by running
-.BR "uname -s" .
+.BR hostname , " h
+Valid if the value matches the short hostname.
+Hostname is calculated by running
+.BR "uname -n" ,
+and trimming off any domain.
 .TP
 .BR class , " c
 Valid if the value matches the
@@ -484,16 +501,41 @@ Class must be manually set using
 See the CONFIGURATION section for more details about setting
 .BR local.class .
 .TP
-.BR hostname , " h
-Valid if the value matches the short hostname.
-Hostname is calculated by running
-.BR "hostname" ,
-and trimming off any domain.
+.BR distro , " d
+Valid if the value matches the distro.
+Distro is calculated by running
+.B "lsb_release -si"
+or by inspecting the ID from
+.BR "/etc/os-release" .
+.TP
+.BR distro_family , " f
+Valid if the value matches the distro family.
+Distro family is calculated by inspecting the ID_LIKE line from
+.BR "/etc/os-release" .
+.TP
+.BR os , " o
+Valid if the value matches the OS.
+OS is calculated by running
+.BR "uname -s" .
+.TP
+.BR arch , " a
+Valid if the value matches the architecture.
+Architecture is calculated by running
+.BR "uname -m" .
 .TP
 .B default
 Valid when no other alternate is valid.
+.TP
+.BR extension , " e
+A special "condition" that doesn't affect the selection process. Its purpose is
+instead to allow the alternate file to end with a certain extension to
+e.g. make editors highlight the content properly.
 .LP
 
+.BR NOTE :
+The OS for "Windows Subsystem for Linux" is reported as "WSL", even
+though uname identifies as "Linux".
+
 You may use any number of conditions, in any order.
 An alternate will only be used if ALL conditions are valid.
 For all files managed by yadm's repository or listed in
@@ -553,7 +595,7 @@ If no "##default" version exists and no files have valid conditions, then no
 link will be created.
 
 Links are also created for directories named this way, as long as they have at
-least one yadm managed file within them.
+least one yadm managed file within them (at the top level).
 
 yadm will automatically create these links by default. This can be disabled
 using the
@@ -572,8 +614,9 @@ command. The following sets the class to be "Work".
 
   yadm config local.class Work
 
-Similarly, the values of os, hostname, and user can be manually overridden
-using the configuration options
+Similarly, the values of architecture, os, hostname, and user can be manually
+overridden using the configuration options
+.BR local.arch ,
 .BR local.os ,
 .BR local.hostname ,
 and
@@ -595,6 +638,15 @@ upon
 which is available on most *nix systems. To use this processor,
 specify the value of "default" or just leave the value off (e.g. "##template").
 .TP
+.B ESH
+ESH is a template processor written in POSIX compliant shell. It allows
+executing shell commands within templates. This can be used to reference your
+own configurations within templates, for example:
+
+  <% yadm config mysection.myconfig %>
+
+To use the ESH template processor, specify the value of "esh"
+.TP
 .B j2cli
 To use the j2cli Jinja template processor, specify the value of "j2"  or
 "j2cli".
@@ -612,24 +664,37 @@ to create or overwrite files.
 
 During processing, the following variables are available in the template:
 
- Default         Jinja           Description
- -------------   -------------   --------------------------
- yadm.class      YADM_CLASS      Locally defined yadm class
- yadm.distro     YADM_DISTRO     lsb_release -si
- yadm.hostname   YADM_HOSTNAME   hostname (without domain)
- yadm.os         YADM_OS         uname -s
- yadm.user       YADM_USER       id -u -n
- yadm.source     YADM_SOURCE     Template filename
+ Default              Jinja or ESH         Description
+ -------------        -------------        ----------------------------
+ yadm.arch            YADM_ARCH            uname -m
+ yadm.class           YADM_CLASS           Last locally defined class
+ yadm.classes         YADM_CLASSES         All classes
+ yadm.distro          YADM_DISTRO          lsb_release -si
+ yadm.distro_family   YADM_DISTRO_FAMILY   ID_LIKE from /etc/os-release
+ yadm.hostname        YADM_HOSTNAME        uname -n (without domain)
+ yadm.os              YADM_OS              uname -s
+ yadm.source          YADM_SOURCE          Template filename
+ yadm.user            YADM_USER            id -u -n
+ env.VAR                                   Environment variable VAR
+
+.BR NOTE :
+The OS for "Windows Subsystem for Linux" is reported as "WSL", even
+though uname identifies as "Linux".
+
+.BR NOTE :
+If lsb_release is not available, DISTRO will be the ID specified in
+/etc/os-release.
 
 Examples:
 
 .I whatever##template
 with the following content
 
-  {% if yadm.user == 'harvey' %}
+  {% if yadm.user == "harvey" %}
   config={{yadm.class}}-{{yadm.os}}
   {% else %}
   config=dev-whatever
+  {% include "whatever.extra" %}
   {% endif %}
 
 would output a file named
@@ -638,9 +703,12 @@ with the following content if the user is "harvey":
 
   config=work-Linux
 
-and the following otherwise:
+and the following otherwise (if
+.I whatever.extra
+contains admin=false):
 
   config=dev-whatever
+  admin=false
 
 An equivalent Jinja template named
 .I whatever##template.j2
@@ -650,8 +718,20 @@ would look like:
   config={{YADM_CLASS}}-{{YADM_OS}}
   {% else -%}
   config=dev-whatever
+  {% include 'whatever.extra' %}
   {% endif -%}
 
+An equivalent ESH templated named
+.I whatever##template.esh
+would look like:
+
+  <% if [ "$YADM_USER" = "harvey" ]; then -%>
+  config=<%= $YADM_CLASS %>-<%= $YADM_OS %>
+  <% else -%>
+  config=dev-whatever
+  <%+ whatever.extra %>
+  <% fi -%>
+
 .SH ENCRYPTION
 
 It can be useful to manage confidential files, like SSH or GPG keys, across
@@ -659,9 +739,15 @@ multiple systems. However, doing so would put plain text data into a Git
 repository, which often resides on a public system. yadm can make it easy to
 encrypt and decrypt a set of files so the encrypted version can be maintained
 in the Git repository.
-This feature will only work if the
+This feature will only work if a supported tool is available.
+Both
 .BR gpg (1)
-command is available.
+and
+.BR openssl (1)
+are supported.
+gpg is used by default, but openssl can be configured with the
+.I yadm.cipher
+configuration.
 
 To use this feature, a list of patterns must be created and saved as
 .IR $HOME/.config/yadm/encrypt .
@@ -675,17 +761,19 @@ For example:
     .gnupg/*.gpg
 .RE
 
-Standard filename expansions (*, ?, [) are supported. Other shell expansions
-like brace and tilde are not supported. Spaces in paths are supported, and
-should not be quoted. If a directory is specified, its contents will be
-included, but not recursively. Paths beginning with a "!" will be excluded.
+Standard filename expansions (*, ?, [) are supported.
+If you have Bash version 4, you may use "**" to match all subdirectories.
+Other shell expansions like brace and tilde are not supported.
+Spaces in paths are supported, and should not be quoted.
+If a directory is specified, its contents will be included, but not recursively.
+Paths beginning with a "!" will be excluded.
 
 The
 .B yadm encrypt
 command will find all files matching the patterns, and prompt for a password. Once a
 password has confirmed, the matching files will be encrypted and saved as
-.IR $HOME/.config/yadm/files.gpg .
-The patterns and files.gpg should be added to the yadm repository so they are
+.IR $HOME/.local/share/yadm/archive .
+The "encrypt" and "archive" files should be added to the yadm repository so they are
 available across multiple systems.
 
 To decrypt these files later, or on another system run
@@ -715,6 +803,22 @@ This can be disabled using the
 .I yadm.auto-exclude
 configuration.
 
+.B Using transcrypt or git-crypt
+
+A completely separate option for encrypting data is to install and use
+transcrypt or git-crypt.
+Once installed, you can use these tools by running
+.B "yadm transcrypt"
+or
+.BR "yadm git-crypt" .
+These tools enables transparent encryption and decryption of files in a git
+repository. See the following web sites for more information:
+
+- https://github.com/elasticdog/transcrypt
+
+- https://github.com/AGWA/git-crypt
+.LP
+
 .SH PERMISSIONS
 
 When files are checked out of a Git repository, their initial permissions are
@@ -722,7 +826,7 @@ dependent upon the user's umask. Because of this, yadm will automatically
 update the permissions of some file paths. The "group" and "others" permissions
 will be removed from the following files:
 
-.RI - " $HOME/.config/yadm/files.gpg
+.RI - " $HOME/.local/share/yadm/archive
 
 - All files matching patterns in
 .I $HOME/.config/yadm/encrypt
@@ -798,7 +902,8 @@ The command which triggered the hook
 The exit status of the yadm command
 .TP
 .B YADM_HOOK_FULL_COMMAND
-The yadm command with all command line arguments
+The yadm command with all command line arguments (parameters are space
+delimited, and any space, tab or backslash will be escaped with a backslash)
 .TP
 .B YADM_HOOK_REPO
 The path to the yadm repository
@@ -817,12 +922,25 @@ is defined as a fully qualified path, this directory will be
 Otherwise it will be
 .IR "$HOME/.config/yadm" .
 
+Similarly, yadm's data files are relative to the "yadm data directory".
+yadm uses the "XDG Base Directory Specification" to determine this directory.
+If the environment variable
+.B $XDG_DATA_HOME
+is defined as a fully qualified path, this directory will be
+.IR "$XDG_DATA_HOME/yadm" .
+Otherwise it will be
+.IR "$HOME/.local/share/yadm" .
+
 The following are the default paths yadm uses for its own data.
 Most of these paths can be altered using universal options.
 See the OPTIONS section for details.
 .TP
 .I $HOME/.config/yadm
-The yadm directory. By default, all data yadm stores is relative to this
+The yadm directory. By default, all configs yadm stores is relative to this
+directory.
+.TP
+.I $HOME/.local/share/yadm
+The yadm data directory. By default, all data yadm stores is relative to this
 directory.
 .TP
 .I $YADM_DIR/config
@@ -833,13 +951,13 @@ This is a directory to keep "alternate files" without having them side-by-side
 with the resulting symlink or processed template. Alternate files placed in
 this directory will be created relative to $HOME instead.
 .TP
-.I $YADM_DIR/repo.git
+.I $YADM_DATA/repo.git
 Git repository used by yadm.
 .TP
 .I $YADM_DIR/encrypt
 List of globs used for encrypt/decrypt
 .TP
-.I $YADM_DIR/files.gpg
+.I $YADM_DATA/archive
 All files encrypted with
 .B yadm encrypt
 are stored in this file.
@@ -864,14 +982,14 @@ Initial push of master to origin
 .B echo ".ssh/*.key" >> $HOME/.config/yadm/encrypt
 Add a new pattern to the list of encrypted files
 .TP
-.B yadm encrypt ; yadm add ~/.config/yadm/files.gpg ; yadm commit
+.B yadm encrypt ; yadm add ~/.local/share/yadm/archive ; yadm commit
 Commit a new set of encrypted files
 
 .SH REPORTING BUGS
 
 Report issues or create pull requests at GitHub:
 
-https://github.com/TheLocehiliosan/yadm/issues
+https://github.com/yadm-dev/yadm/issues
 
 .SH AUTHOR
 
@@ -881,5 +999,8 @@ Tim Byrne <sultan@locehilios.com>
 
 .BR git (1),
 .BR gpg (1)
+.BR openssl (1)
+.BR transcrypt (1)
+.BR git-crypt (1)
 
 https://yadm.io/

yadm.md

@@ -1,6 +1,4 @@
 
-
-
 ## NAME
        yadm - Yet Another Dotfiles Manager
 
@@ -24,35 +22,39 @@
 
        yadm encrypt
 
-       yadm enter
-
        yadm decrypt [-l]
 
        yadm alt
 
        yadm perms
 
-       yadm upgrade
+       yadm enter [ command ]
+
+       yadm git-crypt [ options ]
+
+       yadm transcrypt [ options ]
+
+       yadm upgrade [-f]
 
        yadm introspect category
 
 
 ## DESCRIPTION
-       yadm is a tool for managing a collection of files across multiple  com-
-       puters,  using  a  shared Git repository.  In addition, yadm provides a
-       feature to select alternate versions of files for  particular  systems.
-       Lastly,  yadm  supplies the ability to manage a subset of secure files,
+       yadm  is a tool for managing a collection of files across multiple com‐
+       puters, using a shared Git repository.  In addition,  yadm  provides  a
+       feature  to  select alternate versions of files for particular systems.
+       Lastly, yadm supplies the ability to manage a subset of  secure  files,
        which are encrypted before they are included in the repository.
 
 
 ## COMMANDS
        git-command or git-alias
-              Any command not internally handled by yadm is passed through  to
-              git(1).   Git commands or aliases are invoked with the yadm man-
+              Any  command not internally handled by yadm is passed through to
+              git(1).  Git commands or aliases are invoked with the yadm  man‐
               aged repository.  The working directory for Git commands will be
               the configured work-tree (usually $HOME).
 
-              Dotfiles  are  managed by using standard git commands; add, com-
+              Dotfiles  are  managed by using standard git commands; add, com‐
               mit, push, pull, etc.
 
               The config command is not passed directly through.  Instead  use
@@ -60,91 +62,99 @@
 
        alt    Create  symbolic  links  and  process  templates for any managed
               files matching the naming rules described in the ALTERNATES  and
-              TEMPLATES  sections.  It is usually unnecessary to run this com-
+              TEMPLATES  sections.  It is usually unnecessary to run this com‐
               mand, as yadm automatically  processes  alternates  by  default.
-              This  automatic behavior can be disabled by setting the configu-
+              This  automatic behavior can be disabled by setting the configu‐
               ration yadm.auto-alt to "false".
 
        bootstrap
               Execute $HOME/.config/yadm/bootstrap if it exists.
 
        clone url
-              Clone a remote repository for tracking dotfiles.  After the con-
-              tents  of  the remote repository have been fetched, a "merge" of
-              origin/master is attempted.   If  there  are  conflicting  files
-              already  present  in  the  work-tree,  this  merge will fail and
-              instead a "reset" of origin/master will be done, followed  by  a
-              "stash". This "stash" operation will preserve the original data.
+              Clone a remote repository for tracking dotfiles.  After the con‐
+              tents of the remote repository have been fetched, a "check  out"
+              of  the remote HEAD branch is attempted.  If there are conflict‐
+              ing files already present in the work-tree,  the  local  version
+              will  be  left  unmodified and you'll have to review and resolve
+              the difference.
 
-              You can review the stashed conflicts by running the command
-
-                     yadm stash show -p
-
-              from within your $HOME directory. If you  want  to  restore  the
-              stashed data, you can run
-
-                     yadm stash apply
-              or
-                     yadm stash pop
-
-              The  repository  is  stored  in $HOME/.config/yadm/repo.git.  By
-              default, $HOME will be used as the work-tree, but  this  can  be
+              The repository is  stored  in  $HOME/.local/share/yadm/repo.git.
+              By default, $HOME will be used as the work-tree, but this can be
               overridden  with the -w option.  yadm can be forced to overwrite
               an existing repository by providing the -f option.  If you  want
-              to  use  a  branch  other than origin/master, you can specify it
-              using the -b option.  By default yadm will ask the user  if  the
-              bootstrap  program  should  be  run  (if it exists). The options
-              --bootstrap or --no-bootstrap will either force the bootstrap to
-              be  run,  or  prevent  it  from being run, without prompting the
-              user.
+              to  use a branch other than the remote HEAD branch you can spec‐
+              ify it using the -b option.  By default yadm will ask  the  user
+              if  the  bootstrap program should be run (if it exists). The op‐
+              tions --bootstrap or --no-bootstrap will either force the  boot‐
+              strap to be run, or prevent it from being run, without prompting
+              the user.
 
-       config This command manages  configurations  for  yadm.   This  command
-              works  exactly  they way git-config(1) does.  See the CONFIGURA-
-              TION section for more details.
+       config This  command  manages  configurations  for  yadm.  This command
+              works exactly the way git-config(1) does.  See the CONFIGURATION
+              section for more details.
 
        decrypt
-              Decrypt all files stored in $HOME/.config/yadm/files.gpg.  Files
-              decrypted  will be relative to the configured work-tree (usually
-              $HOME).  Using the -l option will list the files stored  without
-              extracting them.
+              Decrypt all  files  stored  in  $HOME/.local/share/yadm/archive.
+              Files  decrypted  will  be  relative to the configured work-tree
+              (usually $HOME).  Using the -l option will list the files stored
+              without extracting them.
 
        encrypt
-              Encrypt  all  files  matching  the patterns found in $HOME/.con-
-              fig/yadm/encrypt.  See the ENCRYPTION section for more  details.
+              Encrypt all files matching the  patterns  found  in  $HOME/.con‐
+              fig/yadm/encrypt.  See the ENCRYPTION section for more details.
 
        enter  Run  a  sub-shell with all Git variables set. Exit the sub-shell
               the same way you leave  your  normal  shell  (usually  with  the
               "exit"  command).  This sub-shell can be used to easily interact
               with your yadm repository using "git" commands.  This  could  be
-              useful  if  you  are  using  a tool which uses Git directly. For
-              example, Emacs Tramp and Magit can manage files  by  using  this
-              configuration:
+              useful  if you are using a tool which uses Git directly, such as
+              tig, vim-fugitive, git-cola, etc.
+
+              Optionally, you can provide a command after "enter", and instead
+              of invoking your shell, that command will be run with all of the
+              Git variables exposed to the command's environment.
+
+              Emacs Tramp and Magit can manage files by using this  configura‐
+              tion:
+
                   (add-to-list 'tramp-methods
                        '("yadm"
                          (tramp-login-program "yadm")
                          (tramp-login-args (("enter")))
+                         (tramp-login-env (("SHELL") ("/bin/sh")))
                          (tramp-remote-shell "/bin/sh")
                          (tramp-remote-shell-args ("-c"))))
 
+              With this config, use (magit-status "/yadm::").
+
+       git-crypt options
+              If  git-crypt  is installed, this command allows you to pass op‐
+              tions directly to git-crypt, with the environment configured  to
+              use the yadm repository.
+
+              git-crypt enables transparent encryption and decryption of files
+              in  a git repository.  You can read https://github.com/AGWA/git-
+              crypt for details.
+
        gitconfig
-              Pass  options to the git config command. Since yadm already uses
-              the config command to manage its own configurations,  this  com-
-              mand is provided as a way to change configurations of the repos-
+              Pass options to the git config command. Since yadm already  uses
+              the  config  command to manage its own configurations, this com‐
+              mand is provided as a way to change configurations of the repos‐
               itory managed by yadm.  One useful case might  be  to  configure
               the  repository so untracked files are shown in status commands.
               yadm initially configures its repository so that untracked files
-              are  not  shown.   If  you wish use the default Git behavior (to
-              show untracked files and directories), you can remove this  con-
+              are not shown.  If you wish use the  default  Git  behavior  (to
+              show  untracked files and directories), you can remove this con‐
               figuration.
 
                      yadm gitconfig --unset status.showUntrackedFiles
 
        help   Print a summary of yadm commands.
 
-       init   Initialize  a  new, empty repository for tracking dotfiles.  The
-              repository  is  stored   in   $HOME/.config/yadm/repo.git.    By
-              default,  $HOME  will  be used as the work-tree, but this can be
-              overridden with the -w option.  yadm can be forced to  overwrite
+       init   Initialize a new, empty repository for tracking  dotfiles.   The
+              repository  is  stored  in $HOME/.local/share/yadm/repo.git.  By
+              default, $HOME will be used as the work-tree, but  this  can  be
+              overridden  with the -w option.  yadm can be forced to overwrite
               an existing repository by providing the -f option.
 
        list   Print a list of files managed by yadm.  The -a option will cause
@@ -156,57 +166,53 @@
               configs, repo, and switches.  The purpose of introspection is to
               support command line completion.
 
-       perms  Update  permissions as described in the PERMISSIONS section.  It
-              is usually unnecessary to run this command,  as  yadm  automati-
-              cally  processes permissions by default. This automatic behavior
-              can be disabled by setting the configuration yadm.auto-perms  to
+       perms  Update permissions as described in the PERMISSIONS section.   It
+              is  usually  unnecessary  to run this command, as yadm automati‐
+              cally processes permissions by default. This automatic  behavior
+              can  be disabled by setting the configuration yadm.auto-perms to
               "false".
 
-       upgrade
-              Version  2  of  yadm uses a different directory for storing your
-              configurations.  When you start to use version 2 for  the  first
-              time,  you  may  see warnings about moving your data to this new
-              directory.  The easiest way to accomplish  this  is  by  running
-              "yadm  upgrade".   This  command  will start by moving your yadm
-              repo to the new path.  Next it will move any configuration  data
-              to  the new path.  If the configurations are tracked within your
-              yadm repo, this command will "stage" the renaming of those files
-              in the repo's index.  Upgrading will also re-initialize all sub-
-              modules you have added (otherwise they will be broken  when  the
-              repo moves).  After running "yadm upgrade", you should run "yadm
-              status" to review changes which have  been  staged,  and  commit
-              them to your repository.
+       transcrypt options
+              If transcrypt is installed, this command allows you to pass  op‐
+              tions directly to transcrypt, with the environment configured to
+              use the yadm repository.
 
-              You can read https://yadm.io/docs/upgrade_from_1 for more infor-
+              transcrypt  enables  transparent  encryption  and  decryption of
+              files    in    a    git    repository.      You     can     read
+              https://github.com/elasticdog/transcrypt for details.
+
+       upgrade
+              Version  3  of yadm uses a different directory for storing data.
+              When you start to use version 3 for the first time, you may  see
+              warnings about moving your data to this new directory.  The eas‐
+              iest  way to accomplish this is by running "yadm upgrade".  This
+              command will start by moving your yadm repo  to  the  new  path.
+              Next  it  will move any archive data.  If the archive is tracked
+              within your yadm repo, this command will "stage" the renaming of
+              that file in the repo's index.
+
+              Upgrading will attempt to de-initialize and  re-initialize  your
+              submodules. If your submodules cannot be de-initialized, the up‐
+              grade  will fail. The most common reason submodules will fail to
+              de-initialize is because they have local modifications.  If  you
+              are willing to lose the local modifications to those submodules,
+              you  can  use  the -f option with the "upgrade" command to force
+              the de-initialization.
+
+              After running "yadm upgrade", you should run  "yadm  status"  to
+              review  changes  which have been staged, and commit them to your
+              repository.
+
+              You can read https://yadm.io/docs/upgrade_from_2 for more infor‐
               mation.
 
        version
               Print the version of yadm.
 
 
-## COMPATIBILITY
-       Beginning with version 2.0.0, yadm introduced a  couple  major  changes
-       which  may  require you to adjust your configurations.  See the upgrade
-       command for help making those adjustments.
-
-       First, yadm now uses the "XDG Base Directory Specification" to find its
-       configurations.  You  can  read https://yadm.io/docs/upgrade_from_1 for
-       more information.
-
-       Second, the naming conventions for alternate files have  been  changed.
-       You can read https://yadm.io/docs/alternates for more information.
-
-       If you want to retain the old functionality, you can set an environment
-       variable, YADM_COMPATIBILITY=1.  Doing so will  automatically  use  the
-       old  yadm  directory,  and process alternates the same as the pre-2.0.0
-       version. This compatibility mode is deprecated, and will be removed  in
-       future  versions.  This mode exists solely for transitioning to the new
-       paths and naming of alternates.
-
-
 ## OPTIONS
        yadm supports a set of universal options that alter the paths it  uses.
-       The  default paths are documented in the FILES section. Any path speci-
+       The  default paths are documented in the FILES section. Any path speci‐
        fied by these options must be fully qualified. If you  always  want  to
        override  one  or  more  of  these paths, it may be useful to create an
        alias for the yadm command.  For example, the following alias could  be
@@ -215,11 +221,15 @@
               alias yadm='yadm --yadm-repo /alternate/path/to/repo'
 
        The  following  is  the  full  list  of universal options.  Each option
-       should be followed by a fully qualified path.
+       should be followed by a path.
 
        -Y,--yadm-dir
-              Override the yadm directory.  yadm stores its data  relative  to
-              this directory.
+              Override the yadm directory.   yadm  stores  its  configurations
+              relative to this directory.
+
+       --yadm-data
+              Override the yadm data directory.  yadm stores its data relative
+              to this directory.
 
        --yadm-repo
               Override the location of the yadm repository.
@@ -249,68 +259,93 @@
        The following is the full list of supported configurations:
 
        yadm.alt-copy
-              If set to "true", alternate files will be copies instead of sym-
+              If set to "true", alternate files will be copies instead of sym‐
               bolic links.  This might be desirable, because some systems  may
               not properly support symlinks.
 
-              NOTE:  The deprecated yadm.cygwin-copy option used by older ver-
-              sions of yadm has  been  replaced  by  yadm.alt-copy.   The  old
-              option will be removed in the next version of yadm.
-
        yadm.auto-alt
-              Disable  the  automatic  linking described in the section ALTER-
+              Disable  the  automatic  linking described in the section ALTER‐
               NATES. If disabled, you may still run  "yadm  alt"  manually  to
-              create  the alternate links. This feature is enabled by default.
+              create the alternate links. This feature is enabled by default.
 
        yadm.auto-exclude
-              Disable  the  automatic  exclusion  of   patterns   defined   in
+              Disable   the   automatic   exclusion  of  patterns  defined  in
               $HOME/.config/yadm/encrypt.  This feature is enabled by default.
 
        yadm.auto-perms
-              Disable the automatic permission changes described in  the  sec-
+              Disable the automatic permission changes described in  the  sec‐
               tion  PERMISSIONS.   If  disabled,  you may still run yadm perms
-              manually to update permissions.   This  feature  is  enabled  by
-              default.
+              manually to update permissions.  This feature is enabled by  de‐
+              fault.
 
        yadm.auto-private-dirs
               Disable  the automatic creating of private directories described
               in the section PERMISSIONS.
 
+       yadm.cipher
+              Configure which encryption system is used by the encrypt/decrypt
+              commands.  Valid options are "gpg" and "openssl". The default is
+              "gpg".  Detailed information can be found in the section ENCRYP‐
+              TION.
+
        yadm.git-program
-              Specify an alternate  program  to  use  instead  of  "git".   By
-              default, the first "git" found in $PATH is used.
+              Specify an alternate program to use instead of  "git".   By  de‐
+              fault, the first "git" found in $PATH is used.
 
        yadm.gpg-perms
               Disable  the permission changes to $HOME/.gnupg/*.  This feature
               is enabled by default.
 
        yadm.gpg-program
-              Specify an alternate  program  to  use  instead  of  "gpg".   By
-              default, the first "gpg" found in $PATH is used.
+              Specify an alternate program to use instead of  "gpg".   By  de‐
+              fault, the first "gpg" found in $PATH is used.
 
        yadm.gpg-recipient
               Asymmetrically encrypt files with a gpg public/private key pair.
-              Provide a "key ID" to specify which public key to encrypt  with.
-              The  key  must  exist in your public keyrings.  If left blank or
-              not provided, symmetric encryption is used instead.  If  set  to
-              "ASK",  gpg  will  interactively  ask  for  recipients.  See the
-              ENCRYPTION section for more details.  This feature  is  disabled
-              by default.
+              Provide  a "key ID" to specify which public key to encrypt with.
+              The key must exist in your public keyrings.  Multiple recipients
+              can be specified (separated by space).  If  left  blank  or  not
+              provided,  symmetric  encryption  is  used  instead.   If set to
+              "ASK", gpg will interactively ask for recipients.  See  the  EN‐
+              CRYPTION  section for more details.  This feature is disabled by
+              default.
+
+       yadm.openssl-ciphername
+              Specify which cipher should be used by  openssl.   "aes-256-cbc"
+              is used by default.
+
+       yadm.openssl-old
+              Newer  versions  of  openssl  support  the pbkdf2 key derivation
+              function. This is used by default. If this configuration is  set
+              to  "true",  openssl operations will use options compatible with
+              older versions of openssl. If you change this option,  you  will
+              need to recreate your encrypted archive.
+
+       yadm.openssl-program
+              Specify  an  alternate  program to use instead of "openssl".  By
+              default, the first "openssl" found in $PATH is used.
 
        yadm.ssh-perms
               Disable the permission changes to $HOME/.ssh/*.  This feature is
               enabled by default.
 
-       The following  four  "local"  configurations  are  not  stored  in  the
+       The following  five  "local"  configurations  are  not  stored  in  the
        $HOME/.config/yadm/config, they are stored in the local repository.
 
 
        local.class
               Specify  a  class for the purpose of symlinking alternate files.
-              By default, no class will be matched.
+              By default, no class will be matched.  The local host can be as‐
+              signed multiple classes using command:
+
+              yadm config --add local.class <additional-class>
+
+       local.arch
+              Override the architecture for the purpose of  symlinking  alter‐
+              nate files.
 
        local.hostname
-              Override the hostname for the purpose  of  symlinking  alternate
+              Override  the  hostname  for the purpose of symlinking alternate
               files.
 
        local.os
@@ -325,24 +360,24 @@
        to have an automated way of choosing an alternate version of a file for
        a different operating system, host, user, etc.
 
-       yadm will automatically create a symbolic link to the appropriate  ver-
+       yadm will automatically create a symbolic link to the appropriate  ver‐
        sion  of  a  file, when a valid suffix is appended to the filename. The
        suffix contains the conditions that must be met for  that  file  to  be
        used.
 
-       The suffix begins with "##", followed by any number of conditions sepa-
+       The suffix begins with "##", followed by any number of conditions sepa‐
        rated by commas.
 
          ##<condition>[,<condition>,...]
 
-       Each condition is an attribute/value pair, separated by a period.  Some
-       conditions  do  not require a "value", and in that case, the period and
-       value can be omitted. Most attributes can be abbreviated  as  a  single
+       Each  condition is an attribute/value pair, separated by a period. Some
+       conditions do not require a "value", and in that case, the  period  and
+       value  can  be  omitted. Most attributes can be abbreviated as a single
        letter.
 
          <attribute>[.<value>]
 
-       These are the supported attributes, in the order of the weighted prece-
+       These are the supported attributes, in the order of the weighted prece‐
        dence:
 
 
@@ -354,45 +389,62 @@
               Valid  if  the  value matches the current user.  Current user is
               calculated by running id -u -n.
 
-       distro, d
-              Valid if the value matches the distro.  Distro is calculated  by
-              running lsb_release -si.
-
-       os, o  Valid  if the value matches the OS.  OS is calculated by running
-              uname -s.
+       hostname, h
+              Valid if the value matches the short hostname.  Hostname is cal‐
+              culated by running uname -n, and trimming off any domain.
 
        class, c
               Valid if the value matches the local.class configuration.  Class
               must be manually set using yadm config local.class <class>.  See
-              the  CONFIGURATION  section  for  more  details  about   setting
-              local.class.
+              the CONFIGURATION section for more  details  about  setting  lo‐
+              cal.class.
 
-       hostname, h
-              Valid if the value matches the short hostname.  Hostname is cal-
-              culated by running hostname, and trimming off any domain.
+       distro, d
+              Valid  if the value matches the distro.  Distro is calculated by
+              running lsb_release -si or by inspecting the ID from /etc/os-re‐
+              lease.
+
+       distro_family, f
+              Valid if the value matches the distro family.  Distro family  is
+              calculated by inspecting the ID_LIKE line from /etc/os-release.
+
+       os, o  Valid  if the value matches the OS.  OS is calculated by running
+              uname -s.
+
+       arch, a
+              Valid if the value matches the  architecture.   Architecture  is
+              calculated by running uname -m.
 
        default
               Valid when no other alternate is valid.
 
+       extension, e
+              A special "condition" that doesn't affect the selection process.
+              Its purpose is instead to allow the alternate file to end with a
+              certain  extension  to  e.g.  make editors highlight the content
+              properly.
 
-       You may use any number of conditions, in any order.  An alternate  will
-       only  be  used  if  ALL conditions are valid.  For all files managed by
-       yadm's repository or  listed  in  $HOME/.config/yadm/encrypt,  if  they
-       match  this  naming  convention, symbolic links will be created for the
+       NOTE: The OS for "Windows Subsystem for Linux" is  reported  as  "WSL",
+       even though uname identifies as "Linux".
+
+       You  may use any number of conditions, in any order.  An alternate will
+       only be used if ALL conditions are valid.  For  all  files  managed  by
+       yadm's  repository  or  listed  in  $HOME/.config/yadm/encrypt, if they
+       match this naming convention, symbolic links will be  created  for  the
        most appropriate version.
 
        The "most appropriate" version is determined by calculating a score for
        each  version  of  a  file. A template is always scored higher than any
        symlink condition. The number of conditions is the next largest  factor
-       in  scoring.   Files  with  more conditions will always be favored. Any
-       invalid condition will disqualify that file completely.
+       in scoring.  Files with more conditions will always be favored. Any in‐
+       valid condition will disqualify that file completely.
 
        If you don't care to have all versions of alternates stored in the same
        directory  as  the  generated  symlink,  you  can  place  them  in  the
        $HOME/.config/yadm/alt directory. The generated  symlink  or  processed
        template will be created using the same relative path.
 
-       Alternate  linking may best be demonstrated by example. Assume the fol-
+       Alternate  linking may best be demonstrated by example. Assume the fol‐
        lowing files are managed by yadm's repository:
 
          - $HOME/path/example.txt##default
@@ -407,7 +459,7 @@
        If running on a Macbook named "host2", yadm will create a symbolic link
        which looks like this:
 
-       $HOME/path/example.txt    ->    $HOME/path/example.txt##os.Darwin,host-
+       $HOME/path/example.txt    ->    $HOME/path/example.txt##os.Darwin,host‐
        name.host2
 
        However, on another Mackbook named "host3", yadm will create a symbolic
@@ -434,23 +486,23 @@
        then no link will be created.
 
        Links  are also created for directories named this way, as long as they
-       have at least one yadm managed file within them.
+       have at least one yadm managed file within them (at the top level).
 
-       yadm will automatically create these links by default. This can be dis-
-       abled  using  the yadm.auto-alt configuration.  Even if disabled, links
+       yadm will automatically create these links by default. This can be dis‐
+       abled using the yadm.auto-alt configuration.  Even if  disabled,  links
        can be manually created by running yadm alt.
 
-       Class is a special value which is stored locally on each  host  (inside
-       the  local repository). To use alternate symlinks using class, you must
-       set the value of class using the configuration  local.class.   This  is
+       Class  is  a special value which is stored locally on each host (inside
+       the local repository). To use alternate symlinks using class, you  must
+       set  the  value  of class using the configuration local.class.  This is
        set like any other yadm configuration with the yadm config command. The
        following sets the class to be "Work".
 
          yadm config local.class Work
 
-       Similarly, the values of os, hostname, and user can be  manually  over-
-       ridden  using  the  configuration options local.os, local.hostname, and
-       local.user.
+       Similarly, the values of architecture, os, hostname, and  user  can  be
+       manually  overridden  using  the  configuration options local.arch, lo‐
+       cal.os, local.hostname, and local.user.
 
 
 ## TEMPLATES
@@ -461,73 +513,106 @@
        Supported template processors:
 
        default
-              This is yadm's built-in template processor.  This  processor  is
-              very basic, with a Jinja-like syntax. The advantage of this pro-
-              cessor is that it only depends upon awk, which is  available  on
-              most  *nix  systems. To use this processor, specify the value of
-              "default" or just leave the value off (e.g. "##template").
+              This  is  yadm's  built-in template processor. This processor is
+              very basic, with a Jinja-like  syntax.  The  advantage  of  this
+              processor  is  that it only depends upon awk, which is available
+              on most *nix systems. To use this processor, specify  the  value
+              of "default" or just leave the value off (e.g. "##template").
 
-       j2cli  To use the j2cli Jinja template processor, specify the value  of
+       ESH    ESH is a template processor written in POSIX compliant shell. It
+              allows  executing  shell  commands within templates. This can be
+              used to reference your own configurations within templates,  for
+              example:
+
+                <% yadm config mysection.myconfig %>
+
+              To use the ESH template processor, specify the value of "esh"
+
+       j2cli  To  use the j2cli Jinja template processor, specify the value of
               "j2"  or "j2cli".
 
        envtpl To use the envtpl Jinja template processor, specify the value of
               "j2" or "envtpl".
 
+       NOTE: Specifying "j2" as the processor will attempt to use j2cli or en‐
+       vtpl, whichever is available.
 
-       NOTE: Specifying "j2" as the processor will attempt  to  use  j2cli  or
-       envtpl, whichever is available.
-
-       If  the  template  processor  specified is available, templates will be
+       If the template processor specified is  available,  templates  will  be
        processed to create or overwrite files.
 
-       During processing, the following variables are available  in  the  tem-
+       During  processing,  the  following variables are available in the tem‐
        plate:
 
-        Default         Jinja           Description
-        -------------   -------------   --------------------------
-        yadm.class      YADM_CLASS      Locally defined yadm class
-        yadm.distro     YADM_DISTRO     lsb_release -si
-        yadm.hostname   YADM_HOSTNAME   hostname (without domain)
-        yadm.os         YADM_OS         uname -s
-        yadm.user       YADM_USER       id -u -n
-        yadm.source     YADM_SOURCE     Template filename
+        Default              Jinja or ESH         Description
+        -------------        -------------        ----------------------------
+        yadm.arch            YADM_ARCH            uname -m
+        yadm.class           YADM_CLASS           Last locally defined class
+        yadm.classes         YADM_CLASSES         All classes
+        yadm.distro          YADM_DISTRO          lsb_release -si
+        yadm.distro_family   YADM_DISTRO_FAMILY   ID_LIKE from /etc/os-release
+        yadm.hostname        YADM_HOSTNAME        uname -n (without domain)
+        yadm.os              YADM_OS              uname -s
+        yadm.source          YADM_SOURCE          Template filename
+        yadm.user            YADM_USER            id -u -n
+        env.VAR                                   Environment variable VAR
+
+       NOTE: The OS for "Windows Subsystem for Linux" is  reported  as  "WSL",
+       even though uname identifies as "Linux".
+
+       NOTE:  If lsb_release is not available, DISTRO will be the ID specified
+       in /etc/os-release.
 
        Examples:
 
        whatever##template with the following content
 
-         {% if yadm.user == 'harvey' %}
+         {% if yadm.user == "harvey" %}
          config={{yadm.class}}-{{yadm.os}}
          {% else %}
          config=dev-whatever
+         {% include "whatever.extra" %}
          {% endif %}
 
-       would  output  a  file named whatever with the following content if the
+       would output a file named whatever with the following  content  if  the
        user is "harvey":
 
          config=work-Linux
 
-       and the following otherwise:
+       and the following otherwise (if whatever.extra contains admin=false):
 
          config=dev-whatever
+         admin=false
 
-       An equivalent Jinja template  named  whatever##template.j2  would  look
+       An  equivalent  Jinja  template  named whatever##template.j2 would look
        like:
 
          {% if YADM_USER == 'harvey' -%}
          config={{YADM_CLASS}}-{{YADM_OS}}
          {% else -%}
          config=dev-whatever
+         {% include 'whatever.extra' %}
          {% endif -%}
 
+       An equivalent ESH templated  named  whatever##template.esh  would  look
+       like:
+
+         <% if [ "$YADM_USER" = "harvey" ]; then -%>
+         config=<%= $YADM_CLASS %>-<%= $YADM_OS %>
+         <% else -%>
+         config=dev-whatever
+         <%+ whatever.extra %>
+         <% fi -%>
+
 
 ## ENCRYPTION
        It  can  be  useful to manage confidential files, like SSH or GPG keys,
        across multiple systems. However, doing so would put  plain  text  data
        into a Git repository, which often resides on a public system. yadm can
-       make it easy to encrypt and decrypt a set of  files  so  the  encrypted
-       version  can  be  maintained  in the Git repository.  This feature will
-       only work if the gpg(1) command is available.
+       make  it  easy  to  encrypt and decrypt a set of files so the encrypted
+       version can be maintained in the Git  repository.   This  feature  will
+       only work if a supported tool is available.  Both gpg(1) and openssl(1)
+       are  supported.   gpg is used by default, but openssl can be configured
+       with the yadm.cipher configuration.
 
        To use this feature, a list of patterns must be created  and  saved  as
        $HOME/.config/yadm/encrypt.   This  list of patterns should be relative
@@ -536,17 +621,18 @@
                   .ssh/*.key
                   .gnupg/*.gpg
 
-       Standard filename expansions (*,  ?,  [)  are  supported.  Other  shell
-       expansions  like brace and tilde are not supported. Spaces in paths are
-       supported, and should not be quoted. If a directory is  specified,  its
-       contents  will be included, but not recursively. Paths beginning with a
+       Standard filename expansions (*, ?, [) are supported.  If you have Bash
+       version 4, you may use "**" to match all subdirectories.   Other  shell
+       expansions like brace and tilde are not supported.  Spaces in paths are
+       supported,  and should not be quoted.  If a directory is specified, its
+       contents will be included, but not recursively.  Paths beginning with a
        "!" will be excluded.
 
        The yadm encrypt command will find all files matching the patterns, and
-       prompt  for  a  password.  Once  a password has confirmed, the matching
-       files will be encrypted and saved as $HOME/.config/yadm/files.gpg.  The
-       patterns  and  files.gpg should be added to the yadm repository so they
-       are available across multiple systems.
+       prompt for a password. Once a  password  has  confirmed,  the  matching
+       files  will  be encrypted and saved as $HOME/.local/share/yadm/archive.
+       The "encrypt" and "archive" files should be added to the  yadm  reposi‐
+       tory so they are available across multiple systems.
 
        To decrypt these files later, or on another system run yadm decrypt and
        provide  the  correct password.  After files are decrypted, permissions
@@ -559,18 +645,29 @@
        confidential files, even though they are encrypted.
 
        Patterns found in $HOME/.config/yadm/encrypt are automatically added to
-       the  repository's  info/exclude  file  every  time yadm encrypt is run.
-       This is to prevent accidentally committing sensitive data to the repos-
+       the repository's info/exclude file every  time  yadm  encrypt  is  run.
+       This is to prevent accidentally committing sensitive data to the repos‐
        itory.  This can be disabled using the yadm.auto-exclude configuration.
 
+       Using transcrypt or git-crypt
+
+       A  completely separate option for encrypting data is to install and use
+       transcrypt or git-crypt.  Once installed, you can use  these  tools  by
+       running  yadm transcrypt or yadm git-crypt.  These tools enables trans‐
+       parent encryption and decryption of files in a git repository. See  the
+       following web sites for more information:
+
+       - https://github.com/elasticdog/transcrypt
+
+       - https://github.com/AGWA/git-crypt
 
 ## PERMISSIONS
-       When files are checked out of a Git repository, their  initial  permis-
-       sions  are  dependent upon the user's umask. Because of this, yadm will
-       automatically update the permissions of some file  paths.  The  "group"
+       When  files  are checked out of a Git repository, their initial permis‐
+       sions are dependent upon the user's umask. Because of this,  yadm  will
+       automatically  update  the  permissions of some file paths. The "group"
        and "others" permissions will be removed from the following files:
 
-       - $HOME/.config/yadm/files.gpg
+       - $HOME/.local/share/yadm/archive
 
        - All files matching patterns in $HOME/.config/yadm/encrypt
 
@@ -578,11 +675,11 @@
 
        - The GPG directory and files, .gnupg/*
 
-       yadm will automatically update permissions by default. This can be dis-
-       abled using the yadm.auto-perms configuration. Even if  disabled,  per-
-       missions  can  be  manually  updated  by  running yadm perms.  The .ssh
-       directory processing can be disabled using the yadm.ssh-perms  configu-
-       ration.  The  .gnupg  directory  processing  can  be disabled using the
+       yadm will automatically update permissions by default. This can be dis‐
+       abled using the yadm.auto-perms configuration. Even if  disabled,  per‐
+       missions  can  be manually updated by running yadm perms.  The .ssh di‐
+       rectory processing can be disabled using the yadm.ssh-perms  configura‐
+       tion.  The  .gnupg  directory  processing  can  be  disabled  using the
        yadm.gpg-perms configuration.
 
        When cloning a repo which includes data in a .ssh or .gnupg  directory,
@@ -592,26 +689,26 @@
 
        When running a Git command and .ssh or .gnupg directories do not exist,
        yadm will create those directories with mask 0700 prior to running  the
-       Git command. This can be disabled using the yadm.auto-private-dirs con-
+       Git command. This can be disabled using the yadm.auto-private-dirs con‐
        figuration.
 
 
 ## HOOKS
-       For every command yadm supports, a  program  can  be  provided  to  run
-       before  or  after  that command. These are referred to as "hooks". yadm
-       looks for hooks in the directory $HOME/.config/yadm/hooks.   Each  hook
+       For  every  command yadm supports, a program can be provided to run be‐
+       fore or after that command. These are  referred  to  as  "hooks".  yadm
+       looks  for  hooks in the directory $HOME/.config/yadm/hooks.  Each hook
        is named using a prefix of pre_ or post_, followed by the command which
-       should trigger the hook. For example, to create a  hook  which  is  run
-       after  every  yadm  pull command, create a hook named post_pull.  Hooks
-       must have the executable file permission set.
+       should trigger the hook. For example, to create a hook which is run af‐
+       ter every yadm pull command, create a hook named post_pull.  Hooks must
+       have the executable file permission set.
 
        If a pre_ hook is defined, and the hook terminates with a non-zero exit
-       status,  yadm  will  refuse  to run the yadm command. For example, if a
-       pre_commit hook is defined, but that command ends with a non-zero  exit
-       status,  the  yadm commit will never be run. This allows one to "short-
+       status, yadm will refuse to run the yadm command.  For  example,  if  a
+       pre_commit  hook is defined, but that command ends with a non-zero exit
+       status, the yadm commit will never be run. This allows one  to  "short-
        circuit" any operation using a pre_ hook.
 
-       Hooks have the following environment variables  available  to  them  at
+       Hooks  have  the  following  environment variables available to them at
        runtime:
 
        YADM_HOOK_COMMAND
@@ -621,7 +718,9 @@
               The exit status of the yadm command
 
        YADM_HOOK_FULL_COMMAND
-              The yadm command with all command line arguments
+              The yadm command with all command line arguments (parameters are
+              space delimited, and any space, tab or backslash will be escaped
+              with a backslash)
 
        YADM_HOOK_REPO
               The path to the yadm repository
@@ -631,36 +730,46 @@
 
 
 ## FILES
-       All  of  yadm's  configurations  are  relative to the "yadm directory".
-       yadm uses the "XDG Base  Directory  Specification"  to  determine  this
-       directory.   If the environment variable $XDG_CONFIG_HOME is defined as
-       a fully qualified path, this directory will  be  $XDG_CONFIG_HOME/yadm.
+       All of yadm's configurations are  relative  to  the  "yadm  directory".
+       yadm  uses the "XDG Base Directory Specification" to determine this di‐
+       rectory.  If the environment variable $XDG_CONFIG_HOME is defined as  a
+       fully  qualified  path,  this  directory will be $XDG_CONFIG_HOME/yadm.
        Otherwise it will be $HOME/.config/yadm.
 
-       The  following  are the default paths yadm uses for its own data.  Most
+       Similarly, yadm's data files are relative to the "yadm data directory".
+       yadm uses the "XDG Base Directory Specification" to determine this  di‐
+       rectory.   If  the  environment variable $XDG_DATA_HOME is defined as a
+       fully qualified path, this directory will be $XDG_DATA_HOME/yadm.  Oth‐
+       erwise it will be $HOME/.local/share/yadm.
+
+       The following are the default paths yadm uses for its own  data.   Most
        of these paths can be altered using universal options.  See the OPTIONS
        section for details.
 
        $HOME/.config/yadm
-              The yadm directory. By default, all data yadm stores is relative
-              to this directory.
+              The yadm directory. By default, all configs yadm stores is rela‐
+              tive to this directory.
+
+       $HOME/.local/share/yadm
+              The  yadm  data  directory.  By default, all data yadm stores is
+              relative to this directory.
 
        $YADM_DIR/config
               Configuration file for yadm.
 
        $YADM_DIR/alt
               This is a directory to keep  "alternate  files"  without  having
-              them  side-by-side  with the resulting symlink or processed tem-
+              them  side-by-side  with the resulting symlink or processed tem‐
               plate. Alternate files placed in this directory will be  created
               relative to $HOME instead.
 
-       $YADM_DIR/repo.git
+       $YADM_DATA/repo.git
               Git repository used by yadm.
 
        $YADM_DIR/encrypt
               List of globs used for encrypt/decrypt
 
-       $YADM_DIR/files.gpg
+       $YADM_DATA/archive
               All files encrypted with yadm encrypt are stored in this file.
 
 
@@ -680,14 +789,14 @@
        echo .ssh/*.key >> $HOME/.config/yadm/encrypt
               Add a new pattern to the list of encrypted files
 
-       yadm encrypt ; yadm add ~/.config/yadm/files.gpg ; yadm commit
+       yadm encrypt ; yadm add ~/.local/share/yadm/archive ; yadm commit
               Commit a new set of encrypted files
 
 
 ## REPORTING BUGS
        Report issues or create pull requests at GitHub:
 
-       https://github.com/TheLocehiliosan/yadm/issues
+       https://github.com/yadm-dev/yadm/issues
 
 
 ## AUTHOR
@@ -695,9 +804,7 @@
 
 
 ## SEE ALSO
-       git(1), gpg(1)
+       git(1), gpg(1) openssl(1) transcrypt(1) git-crypt(1)
 
        https://yadm.io/
 
-
-

yadm.spec

@@ -1,18 +1,13 @@
 %{!?_pkgdocdir: %global _pkgdocdir %{_docdir}/%{name}-%{version}}
 Name: yadm
 Summary: Yet Another Dotfiles Manager
-Version: 2.0.1
+Version: 3.3.0
 Group: Development/Tools
 Release: 1%{?dist}
 URL: https://yadm.io
 License: GPL-3.0-only
 Requires: bash
 Requires: git
-%if 0%{?fedora} || 0%{?rhel_version} || 0%{?centos_version} >= 700
-Requires: /usr/bin/hostname
-%else
-Requires: /bin/hostname
-%endif
 
 Source: %{name}.tar.gz
 BuildRoot: %{_tmppath}/%{name}-%{version}-build
@@ -34,7 +29,7 @@ encrypted before they are included in the repository.
 
 # this is done to allow paths other than yadm-x.x.x (for example, when building
 # from branches instead of release tags)
-cd *yadm-*
+test -f yadm || cd *yadm-*
 
 %{__mkdir} -p %{buildroot}%{_bindir}
 %{__cp}  yadm %{buildroot}%{_bindir}