GitHub - jjba23/sift: Lisp, Scheme, Racket and Emacs Lisp formatter, linter and LSP (language server)
Lisp, Scheme, Racket and Emacs Lisp formatter, linter and LSP (language server) - jjba23/sift
Lisp, Scheme, Racket and Emacs Lisp formatter, linter and LSP (language server) - jjba23/sift
an idempotent formatter, a rule-based linter and a language server for Scheme, Racket, Emacs Lisp and Common Lisp
sift reads a Lisp source file, reports what is ill-formed in it, and rewrites
its whitespace. The rewrite is a total function on text and a fixed point:
formatting a formatted file returns it unchanged, and a comment, a blank line,
a #; datum comment or a #. read-time form appears in the output in the place
it was in the input. Both properties are asserted over every fixture in the
suite, so neither is a promise that expires quietly.
Four dialects are read, each by the rules of its own report rather than by a
compromise between them: Scheme, Racket, Emacs Lisp and Common Lisp. A square
bracket is a vector in one of them and a clause in the next, sift does not
average the two, and a Racket module is read from the #lang line that opens
it rather than from the extension of the file it is in.
Install it with guix install sift, nix run, Homebrew (a tap in this
repository), or clone the repository and run ./bin/sift
version. The implementation is Guile 3.0 and the standard library, with no C
dependency and no build step, because a Scheme program has none; sift doctor
reports the interpreter it runs on, the library it loaded and the rules it
found.
The same program is a language server (sift lsp). The findings, the fixes and
the formatting are computed once and delivered either as text on standard
output or as LSP responses, so Emacs through Eglot, Neovim and any other client
of the protocol are told exactly what a pipeline is told.
It ships as one executable and one Emacs package, under the MIT license. The premise is the ordinary one: a Lisp program is a datum with a read syntax, and a formatter that treats it as such can be exact and cheap at the same time.
Contributions are welcome, and the two pages for them are short:
CONTRIBUTING.md is the five minutes from git clone to a green suite, and
doc/contributing.org is what a new rule or a new dialect has to reach –
which the suite then checks for you.
cond clauses are written with square
brackets and whose build files are gexps, want a formatter that agrees with
the tree next door instead of re-deciding it.lexical-binding,
interactive and provide rules in Flymake and in CI, from one program
rather than three.LOOP keyword style and
dangling parentheses are convention, and are preserved rather than
normalised away.--format=json,
--format=sarif, GitHub annotations, or sift lsp.#; datum comments,
#. read-time forms and #<...> objects return exactly as they were read.
Formatting is a fixed point: the second run changes nothing. The suite
asserts preservation and idempotence over every fixture.#| ... |# block comments, #; datum
comments, package prefixes, LOOP and docstring conventions follow the
dialect’s own report rather than a compromise. So does a square bracket,
which is a vector in Emacs Lisp and a clause in Scheme, #{ ... }#, which is
a symbol and only Guile reads, and a #lang line, which opens a Racket
module and means nothing anywhere else. Guile is the dialect sift is
written in and the one its reader knows most closely; the other three are
read from their specifications.format prints, --write writes, lint reports, lint --fix
fixes, and ci asks the formatting question and the lint question in one
pass without touching the tree.SIFT012), a sentence, the dialects it applies to, an example
and its own test file. sift explain RULE reads the reasoning out of the
program, and doc/rules.org is generated from the rule records, so the
book cannot drift from the code it describes.;; sift:disable-next-line keyword-arguments switches one rule off for one
line, disable for the rest of the form it sits in, and enable turns it
back on. Suppression is line-oriented and form-scoped by design: an
exception list in a configuration file is a list that no review ever reaches,
and a range comment outlives the code it was written for.sift lsp speaks LSP over standard
input and output: diagnostics, quickfixes and formatting, go to definition
and references for all four dialects – including the names the standard
library and its installed packages define, the documentation an
implementation carries for the names it compiled in – written as a source
file in that dialect, so a jump lands in code and not in a list of prose –
and the compressed sources most Emacs installations ship – an outline,
completion, inlay hints
that name a call’s arguments, a hover that describes a symbol or the rule
under it, import organization, a file rename that keeps its module in step,
and a second opinion from the dialect’s own compiler when a buffer is saved.
The standard library sift can index is the one of Scheme, Racket, Emacs Lisp
and Common Lisp: Racket ships collects and pkgs as source, so its library
is read like a project’s, and a name that only the compiled #%kernel
defines is the one it cannot answer.:make and compilation-mode parse, JSON for editors that want structure,
SARIF 2.1.0 for code scanning, GitHub annotations for a pull request, and
null for a script that cares only about the exit status.sift config show --verbose
reporting which layer won.sift install-hooks
writes the pre-commit hook that runs ci; sift doctor reports which Guile
it runs on, which library it loaded, how many rules it found and which
configuration layer won; sift list-checkers prints the catalogue as a table
and sift explain prints one rule’s reasoning, both read out of the rule
records rather than out of a document that may have drifted from them.One decision carries most of the list: a Lisp file is read as a tree of forms carrying source positions, not as lines of text, and the formatter prints that tree back. Preservation, a fix that replaces one range, and a diff whose hunks are the change all follow from it. That the reader and the printer are inverse operations is the oldest property Lisp has; the work here is to keep them inverse while the whitespace moves.
Guile 3.0 is the only requirement. The launcher inside a checkout runs the program, so a clone is enough:
git clone https://github.com/jjba23/sift.git cd sift ./bin/sift version
A file that needs work:
$ cat square.scm (define (square x) (* x x)) $ sift check square.scm --- square.scm +++ square.scm @@ -1,2 +1,2 @@ (define (square x) -(* x x)) + (* x x)) $ echo $? 1
check returns 0 if the formatting is good. If not, it prints a patch to stdout and returns 1. It never modifies your files.
To make the change:
$ sift format --write square.scm $ sift check square.scm; echo $? 0
A file with something to say:
$ cat small.scm
(define (go)
(if (ready?)
(proceed)))
$ sift lint small.scm
small.scm:2:4: warning: [if-without-else] `if` has no else branch; `when` says it once
1 file checked, 1 warning, 0 errors
Each line has the shape file:line:column: severity: [rule] sentence, which a
compilation buffer, a CI log and grep already parse. Suggestions are findings
too, and a deeply nested body is reported rather than rewritten:
$ sift lint messy.scm messy.scm:5:3: info: [wrap-in-let] this body form nests 10 deep; bind part of it messy.scm:6:5: info: [wrap-in-let] this body form nests 9 deep; bind part of it messy.scm:7:7: info: [wrap-in-let] this body form nests 8 deep; bind part of it messy.scm:8:9: info: [wrap-in-let] this body form nests 7 deep; bind part of it 1 file checked, 0 warnings, 0 errors
To read what a rule means, ask the program that holds it:
$ sift explain if-without-else CODE SIFT012 NAME if-without-else DESCRIPTION `if` with no else branch, where `when` expresses the intent DIALECTS scheme SEVERITY warning FAILING EXAMPLE (if (ready?) (go)) EXPECTED FIX (when (ready?) (go))
When the tool and the environment disagree about what is running, the program can be asked what it sees:
$ sift doctor sift 1.37.0 guile 3.0.11 library /home/joe/Hacking/sift/src rules 95 dialects scheme, racket, elisp, cl Compilers scheme available elisp available cl not found Sources cache /home/joe/.cache/sift/sources downloaded 0 tools curl, tar Configuration project /home/joe/project/sift.yaml Settings that are not the default brackets: square # /home/joe/project/sift.yaml severity: warning # /home/joe/project/sift.yaml Try `sift ci .' to check a tree, or `sift explain RULE' to read a rule.
Two further commands follow from the same records. fix applies the fixes the
rules can state as equivalences and reports what remains:
$ cat messy.scm (define (f) (let ((x 1)) x)) (define (g) (if a b)) $ sift fix messy.scm 1 file checked, 0 warnings, 0 errors $ cat messy.scm (define (f) 1) (define (g) (when a b))
It does not reformat, because formatting belongs to format, and a fix that
moved whitespace would touch every line of the patch it was meant to make
readable. It does tidy imports: the unused ones are removed and the
specifications are sorted, both on by default, and optimize-imports or
sort-imports in sift.yaml turns either off. ci is the same pair of
questions asked by a pipeline, in one command and one status:
$ sift ci . --- ./go.scm +++ ./go.scm @@ -1,3 +1,3 @@ (define (go) - (if (ready?) - (proceed))) + (if (ready?) + (proceed))) ./go.scm:2:6: warning: [if-without-else] `if` has no else branch; `when` says it once 1 file checked, 1 warning, 0 errors $ echo $? 1
At run time sift needs Guile 3.0 or newer and nothing else. Emacs is optional, and only for the editor half.
| What | Needed for | Version |
|---|---|---|
| Guile | the whole tool, at run time | 3.0+ |
| `gzip` | reading a compressed source in navigation | any |
| `curl`, `tar` | fetching a source archive named in `stdlib` | any |
| Emacs | the Emacs package in emacs/sift.el | 28.1+ |
| Guix, Nix or Homebrew | installing as the distribution does | any |
| Vim and Node | running the Vim and VS Code tests only | any |
| Veritas | running the test suite only | dev |
The language server asks whichever of guild, emacs and sbcl is installed
for its second opinion on a buffer; a compiler that is not present is simply
not asked, and compiler-diagnostics: false turns the second opinion off.
sift runs on Linux and macOS. Caches follow the platform: $XDG_CACHE_HOME/sift
when it is set, /Library/Caches/sift on macOS, and $XDG_CACHE_HOME/sift or
/.cache/sift elsewhere. The optional tools above are found on PATH; a
missing one is reported and the feature that needed it is skipped, rather than
the tool refusing to run.
No C dependency and no build step are involved, and no package manager sits in the path: the program is Scheme source, loaded by whichever Guile is present.
git clone https://github.com/jjba23/sift.git cd sift ./bin/sift version
bin/sift is a small POSIX shell launcher: it resolves its own symlink chain,
puts the sibling src/ on Guile’s load path (or $SIFT_LOAD_PATH, if one is
set), and hands the arguments to (sift cli). Where the launcher is not
wanted, the program reduces to this:
#!/usr/bin/env guile !# (use-modules (sift cli)) (exit (sift-main (cdr (program-arguments))))
run with GUILE_LOAD_PATH pointing at src/. For a prefix,
maak install copies the library to $(prefix)/share/guile/site/3.0 and the
launcher to $(prefix)/bin; PREFIX=/usr/local is the default and
maak uninstall is its inverse.
On the first run Guile compiles the library and prints a few
;;; note: ... lines to standard error while it does. That is Guile’s
auto-compilation, not sift: it stops once the cache is warm, and
GUILE_AUTO_COMPILE=0 suppresses it.
This repository is a tap, so macOS gets sift from the formula it carries:
brew tap jjba23/sift https://github.com/jjba23/sift brew install jjba23/sift/sift
The URL is spelled out because the one-argument form looks for a repository
named homebrew-sift, and the formula lives in this one instead.
Guile is the formula’s only dependency, and Homebrew installs it. The
launcher is pointed at that interpreter rather than at whichever Guile a
PATH happens to hold, which is the substitution the Nix and Guix packages
make as well. There is nothing to build: the formula installs the launcher
and the library in the relation a checkout has them, so installing is copying.
brew upgrade sift follows a release, and brew uninstall sift removes what
the formula installed. A release rewrites the tag in the formula’s url, and
the sha256 of the new archive is written afterwards, from the published tag;
development.org has that step.
From a checkout, the shell command builds the package it ships:
guix shell -f guix.scm -- sift version guix build -f guix.scm # or just the store path
To install it as a package, the guix-jointhefreeworld channel carries it:
guix install sift
The split between runtime and build inputs is deliberate. sift is a library plus a launcher, and needs only Guile. Veritas, the test framework, is needed to run the suite and is a native input of the package for that reason, so installing sift on a machine that will never run its tests installs no test framework.
The repository carries its own derivation, so Nix does not need a second expression to build it:
nix-build -E 'with import <nixpkgs> {}; callPackage ./default.nix { }'
./result/bin/sift version
It also carries a flake: the same derivation with the ways in attached — a package, an app, a development shell and a container image:
nix run .#sift -- version # in a checkout nix run git+https://github.com/jjba23/sift -- version nix build .#sift # ./result/bin/sift nix develop # a shell with sift on PATH nix build .#docker # a loadable image, on Linux
The flake builds on Linux and on macOS (x86_64-darwin, aarch64-darwin),
because the closure is Guile and Scheme and nothing else: on a Mac it is the
route for a machine that already has Nix, and the tap is
the shorter one for a machine that does not. The image is the one output that
is Linux only, and on Darwin it is absent rather than broken.
The nix-jointhefreeworld flake carries the same package, built from this
repository’s own default.nix:
nix run git+https://github.com/jjba23/nix-jointhefreeworld#sift -- version
The image holds Debian’s Guile, the library and the launcher, and nothing else: there is no build step inside it, because a Scheme program has none. Build it from the checkout, or pull it:
docker build --tag sift . # or: podman build --tag sift . docker run --rm --volume "$PWD:/src" sift check .
The container’s working directory is /src, so a volume mounted there is the
tree sift reads; the entry point is sift itself, so the arguments after the
image are sift’s arguments. A whole-project run, a patch, a lint and the
language server (on standard input and output) all work the same way:
docker run --rm --volume "$PWD:/src" sift ci . docker run --rm --interactive --volume "$PWD:/src" sift lsp
maak package-docker is the first command with the tags already on, and
maak push-docker-latest / maak push-docker-tag publish it — the same three
tasks maak’s own file has. The image is also a flake output for a machine with
Nix and no engine: nix build .#docker writes a tarball that
docker load < result loads.
A container exists for the other end of the pipe. On macOS no Guile is present that a script can rely on, and no Guix; Docker Desktop is. One image that runs, with a test over it, is cheaper than three install procedures that would each have to be verified on their own.
emacs/sift.el is a single file with no dependencies beyond Emacs 28.1:
(add-to-list 'load-path "/path/to/sift/emacs") (require 'sift)
The file carries the headers a package needs, so M-x package-install-file on
emacs/sift.el also works, for a copy that should live in
package-user-dir.
What is enabled after that is a matter for the project. The Emacs chapter has the two arrangements, with and without Eglot, and the settings that can be changed.
sift version # the number, and that the program starts
sift doctor # the Guile it runs on, the library it loaded, the rules it
# has, and the configuration it decided on
Every command, every option and every exit code is in
doc/cli.org, which is the reference. In brief: sift format, sift lint and
sift check take files or standard input, sift ci is check and lint in one
command for a pipeline, and the status is 0 for clean, 1 for findings and 2 for
a usage error or a file that cannot be read.
Any of the file commands can be watched instead of run once: sift ci --watch,
sift lint --watch src/ and sift format --write --watch . run again whenever
a file the run read changes, until you interrupt them. It watches what the run
read rather than what it was given, so a file created three directories down is
noticed, and it writes its own lines to standard error so a machine format stays
parseable.
The determination is an ordered sequence of tests, and the first that answers wins:
--dialect, or the dialect setting..scm, .ss, .sld, .sls and .sps are Scheme; .rkt,
.rktd and .rktl are Racket; .el is Emacs Lisp; .lisp,
.cl, .asd, .lsp are Common Lisp.guile is Scheme, racket is Racket, sbcl, clisp, ecl
and ccl are Common Lisp, emacs is Emacs Lisp.#lang line is Racket,
(in-package ...) or (defpackage ...) is Common Lisp,
(define-module ...) or (use-modules ...) is Scheme, a
lexical-binding: t header is Emacs Lisp.An editor that has already resolved the language is better informed than any of
this: the language server takes the identifier the client sends (scheme,
racket, emacs-lisp, lisp) and goes by that.
The formatter normalises line endings and spacing, places closing delimiters, computes indentation, aligns contiguous inline comments when asked, and preserves blank lines up to the limits of the style. Nothing else is touched: the position of a line break, the contents of a string and the spelling of an escape belong to the author. The function is deterministic and idempotent, and the suite formats every fixture twice to establish it. doc/formatting.org is the page: the two styles, the three indentation rules, every dialect’s read syntax, and the four things it will not do.
A rule is a record: a name, a code, a description, the dialects it applies to, a severity, a procedure and an example. The driver runs those the dialect selects, adds the reader’s own findings, applies the suppression comments, and emits the result in source order. The catalogue holds 95 of them, listed one by one in doc/rules.org. doc/linting.org covers the rest: what a fix may touch, why a fix is discarded rather than guessed at, and how a rule is silenced for a line or for a form.
The style is not an invention: it is the common practice of the four Lisp
communities, written down where it can be argued with.
doc/style.org is the long form – each choice, the alternative it beat and the
reason – including which of Riastradh’s rules sift keeps and what it
declines, the Emacs Lisp conventions that are really byte-compiler warnings, the
reasons Common Lisp keeps its closers, the square bracket a Scheme cond
clause is written with, and the rules sift declines to run on a Racket file
because their premise is another dialect’s. The settings a project is likely to hold differently
are listed in the configuration page: a project that has settled a
convention should not have to argue with its formatter about it.
emacs/sift.el is a thin package: Emacs carries the text to the executable and
the findings back, and every decision stays in one place. It requires Emacs
28.1 or newer and no external package. There are two ways in – Flymake for the
rules, Eglot for the code actions – and doc/emacs.org is the page:
the Flymake backend and its keymap,
Eglot and the code actions,
every setting in the customization group, and
why sift and TAB can disagree about a third-party macro.
Anything that speaks LSP can have the server, and anything that reads a compiler’s output can have the program. doc/editors.org carries the details and the recipes, and the ready-made ones live in the repository:
| Editor | What to use |
|---|---|
| Neovim | editors/vim/neovim.lua — a vim.lsp.start setup, no plugin |
| Vim | editors/vim/ — :SiftLint, :SiftFormat, :SiftFix, :SiftExplain, and :compiler sift for :make |
| VS Code | editors/vscode/ — an extension with no build step and no dependencies |
| Emacs | emacs/sift.el — Flymake, or Eglot for code actions |
| Helix | formatting on save through sift format --stdin, and the linter in the pipeline |
| the browser | web/ — sift compiled to WebAssembly, so the buffer never leaves the tab; it is live at https://jointhefreeworld.org/sift/ |
| anything else | --format=json, --format=github, or sift lsp |
The human-readable format is file:line:column: severity: [rule] message,
which Vim’s :make, Emacs’s compilation-mode and most CI log viewers already
parse.
web/ is the two jobs that are a function of a string — formatting and linting — compiled to WebAssembly with Guile Hoot, with an editor page around it. The same reader, the same formatter and all 95 rules run in the tab, and no input leaves it, because there is nowhere for it to go. `maak web` asks whether it compiles and `maak web-check` whether it runs — the second runs the module in Hoot’s own virtual machine and asks every operation — because a module miscompiled by an optimizer compiles, loads, and dies later.
It is live at https://jointhefreeworld.org/sift/: an editor on a page, with the dialect, the style and the severity as controls, the findings in a panel that links back to the source, an outline, a completion popup, inlay hints that name a call’s arguments, and a rules browser built from the same catalogue the command line prints. The buffer opens on an example of the dialect that is selected – Scheme, Racket, Emacs Lisp or Common Lisp – so the page shows the language before it is asked anything, and the theme follows the system or the button, whichever came last.
The evident way to put a Lisp formatter in a browser is to write it again in
JavaScript, and that is a second formatter, which is a second style: the two
agree until the first file that exercises them, at which point the project has to
decide which of its own tools it believes. The source of truth therefore stayed
where it was. The tab runs the reader, the formatter, the lint driver and every
rule that bin/sift runs, because they are the same modules.
The whole contract between the page and the Scheme is one procedure of one string to one string:
(web-request "{\"op\":\"lint\",\"source\":\"(if a b)\"}")
;; => "{\"ok\":true,\"result\":{\"warnings\":[…],\"counts\":{…}}}"
Everything the tab can do is that procedure: format, check, lint, fix, the rule
catalogue, and the four editor questions (hover, go to definition, completion,
outline), which are answered by the language server’s own analysis code
against an empty workspace. Everything it cannot do requires a filesystem —
no directory to walk, no sift.yaml, no save, no server — and the page reports
nothing there rather than guessing.
Hoot compiles R7RS and a subset of Guile, and the gap between that and what sift
uses is web/compat/. Four Guile libraries have stand-ins: format,
textual-ports, (a tab is one thread, so (sift pool) takes its
sequential path) and ftw (scandir answers nothing, because there are no
directories to walk — which is the truth about a tab and not a guess).
The demanding case was SRFI-13. Twenty-six files in the core import it, Hoot
ships srfi-9, srfi-11 and srfi-14 and no SRFI-13, and Guile’s own
srfi/srfi-13.scm is written with a re-export Hoot does not implement — so a
browser build that reached for it would compile a re-export of names the target
does not define and stop there. The answer is a stand-in of thirteen
procedures, written against (scheme base), and a test that runs the browser’s
copy in a second process and fails unless its format, lint and fix answers are
byte for byte the answers Guile’s library gives. A stand-in that is never
compared is a second implementation with no reason to agree with the first.
Four things could not be shimmed, and the source changed instead, in ways that
leave a native build no worse off: (sift strings) carries the four string
procedures R7RS leaves out, rather than importing a library the target may not
have; three (char-set …) call sites became predicates, because a character set
is an object R7RS does not have; one procedure takes its port in the body rather
than as a keyword default that would be evaluated for nothing; and the
filesystem question in (sift imports) asks whether this Scheme has stat and
scandir rather than who is compiling it.
Nothing in src/sift/ names the browser, Wasm or Hoot. The dependency runs one
way, and eight tests hold it that way: the core does not know that a tab exists;
the facade’s dependency on the core is still five modules; every Guile library in
the compiled closure has a stand-in; the SRFI-13 stand-in is the file a build
finds; it answers what Guile’s own library answers, measured in a second process;
the page names no URL from the root of a site; every URL it names carries the
build stamp; and the editor asks for the wasm beside itself rather than at the
root.
The module is 3.4 MB, which is a formatter, a linter and 95 rules on a Wasm GC — a large page by the standards of a web page and a small one by the standards of an editor. Speed is the part of the experiment with numbers attached:
| Buffer | format | lint |
|---|---|---|
| 300 chars | 17 ms | 56 ms |
| 1.2 kB | 52 ms | 145 ms |
| 6 kB | 780 ms | 1.2 s |
A screenful falls below the 300 ms debounce the editor applies to a lint, so the
first two rows never reach the reader; past two kilobytes the cost is
perceptible. There is no JIT, a record is allocated per node, and the reader
allocates four records per character: that is the price of a Scheme on a Wasm
GC, not something the facade can tune away. Where it matters, the request takes
a checkers list, which is how a page asks for less work.
Two questions were being asked. Whether a Scheme program of real size — not a toy, but a formatter with a reader of its own and 95 rules — can be compiled into something a tab runs: it can, with the caveats above, and the porting work amounted to one compatibility directory rather than a rewrite. Whether the seam survives contact: it does, and the test that establishes it runs under Guile in the ordinary suite, with no browser and no Wasm runtime involved.
The boundary is where the design is constrained. A tab has a buffer and no project, so two rules cannot fire and report nothing rather than guessing, and the language server is not in the tab at all: it is a process on standard input and output, which a browser is not. The experiment covers the formatter and the linter, and those are most of what a Lisp file needs while it is being written.
./web/build.sh # writes web/dist/, and finds a Hoot that works maak web-serve # then open http://localhost:8080/ maak web-deploy # or copy it to the directory nginx serves
The build is one command because Hoot is not a library sift ships: it is a
compiler, needed once and never at run time, and locating it is the only
difficulty. Hoot’s compiled modules are bytecode, and bytecode belongs to the
Guile that compiled it, so guix shell guile guile-hoot yields a toolchain that
fails halfway through a compile with an error raised inside Hoot; web/build.sh
therefore compiles a two-line program to establish that its toolchain works, and
asks Guix for guile-next guile-hoot when it does not. web/README.org has the
long form of that, of the two things the server contributes (a .wasm content
type and a trailing-slash redirect) and of the one it imposes (a two-day cache,
which is what the build stamp in every URL is for).
Publishing is a copy into /srv/http/jointhefreeworld.org/sift on the machine
that serves, which is how the rest of that site is published: no ssh, no rsync,
no server to reload, because a file that is present is a file that is served.
The extent of what sift lsp answers – request by request, including the
requests it refuses, its code actions and the two limits that accompany them –
is in doc/language-server.org. The limits bear repeating: a reference is
matched by name within one dialect, because sift expands no macro and follows no
import; and the project is read in the background, so a jump answers from the
files read so far instead of waiting for the rest. A client that advertises
experimental is told how far the reading has got by the sift/indexStatus
notification (uninitialized, scanning, indexing, ready); the states, the
bounds (100 ms for a jump, 200 ms for a reference search) and the batch
size are in
While the project is being read.
One command asks both questions:
sift ci src/ tests/
It checks the formatting, then lints, prints both reports in the format it was asked for, and exits 1 if either found something, 2 if a file could not be read or parsed. It never writes and never fixes: a job that edits the tree it is judging is a job whose verdict means nothing.
There is a reusable workflow, so most repositories need one file and no flags:
# .github/workflows/sift.yml
name: sift
on: [push, pull_request]
jobs:
sift:
uses: jjba23/sift/.github/workflows/sift.yml@v1.37.0
It annotates the diff with what it found and uploads the SARIF document to code
scanning, so a finding that was not fixed becomes an alert in the Security tab
rather than a line in a log that is never scrolled again. It needs no Guix and
no build: the job installs Guile 3.0 from the runner’s packages and runs the
checkout as it stands, and with: install: guix builds it instead for a caller
that wants a store path. The same repository publishes a composite action, for
a job that wants sift as one step among its own:
- uses: jjba23/sift@v1.37.0
with:
paths: src/ tests/
severity: warning
Spelling the step out is also possible: --format=github writes workflow
commands instead of prose, so each finding lands as an annotation on the line of
the diff:
- name: sift run: ./bin/sift ci --format=github src/
::warning file=src/foo.scm,line=3,col=3,title=SIFT012 [if-without-else]::`if` has no else branch; `when` says it once ::error file=src/bar.scm,line=1,title=sift format::src/bar.scm would be reformatted by `sift format --write`
And --format=sarif writes one document for the whole run, which is the shape a
code scanning service consumes:
- run: ./bin/sift ci --format=sarif src/ > sift.sarif
- uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: sift.sarif
sift install-hooks # writes .git/hooks/pre-commit, running `sift ci' sift install-hooks --force # replaces one that is already there
The hook runs whichever sift is on PATH; set SIFT in the hook’s environment
to name another copy. The pre-commit framework is also
supported: .pre-commit-hooks.yaml declares three hooks — sift-format
(rewrites), sift-check (fails) and sift-lint — so a project selects the one
it needs:
repos:
- repo: https://github.com/jjba23/sift
rev: v1.37.0
hooks: [{id: sift-format}, {id: sift-lint}]
A run over a project is a sequence of runs over single files, and the files are
independent, so lint, format, check and ci distribute them over the
processors and print the answers in file order: a parallel run writes exactly the
bytes a sequential run writes. SIFT_JOBS is the only setting involved, and the
two runs that are never parallel are the ones that write the files they read.
The setting, the two exceptions and the reason the output is ordered are in
doc/configuration.org.
A project’s settings are a sift.yaml, or a sift.scm the project has trusted,
read in four layers and validated against one schema: a key that sift does not
have is refused with its position rather than ignored, and
sift config show --verbose reports which layer won.
doc/configuration.org has the whole of it – the discovery rules, the
file-filter, the trust model, and the settings the language server reads out of
the same files.
Every part of the project is tested with the framework its own users already
have – Veritas for the library, ERT for the Emacs package, Vim and Node for the
editor integrations – and none of the suite is a smoke test.
doc/development.org is the page: the maak tasks, what the benchmarks
measure, and what the suite actually asserts. Those tasks are also the
pipeline: a pull request against trunk runs every gate before the branch
lands, and only the push that lands it publishes the browser build
(the pipeline).
src/sift/ is one module per job, and the dependencies run one way: records is
a vocabulary, reader and format use it, the checkers use records and
check/base, lint assembles them, render prints their findings, lsp
delivers the same findings over a protocol, and cli wires the lot to the command
line. Nothing in format or in a checker knows about ports or exit codes. The
module map, and what each module owns, is
doc/development.org.
Six premises the code is measured against: a source file is a datum, not a string; a guarantee that is not a test expires quietly; the author decides what the code says and the tool decides where the whitespace goes; a dialect is a language, not a variant; state what can be decided and report no finding otherwise; and everything a program says should be available as data. doc/design.org states them, and then the decisions that follow from them – a reader of its own, one walk with the answers kept, fuzzy formatting rather than pretty-printing, and a YAML reader small enough to audit.
sift itself does not expand macros, does not type-check, and does not know a
project’s conventions. It reports; the author decides. That is not modesty –
a linter that decided would be useless on the first project it had never seen.
Where the answer can be wrong, the rule says so: wrap-in-let is a
suggestion, keyword-arguments counts, and the thresholds are settings. The
language server can ask the project’s own compiler for the checking sift does
not do – on open and on save, in a thread, on a temporary copy of the buffer
– and publishes what that compiler says about the names, the arity and the
types it can see; compiler-diagnostics: false turns the second opinion off.
The language server answers the requests listed on
the language server page, and answers the others
with “method not found” rather than a guess.
The list of Lisp dialects that deserve a better reader is long, and each item on it is a contribution that can be carried to the end:
sift version with the first section of
sift doctor – the dialect chosen and the rules that ran usually answer the
question on their own. A file that sift refuses to parse is a bug even when
the file is broken: the message is obliged to say where.doc/contributing.org is the checklist behind the two items that reach the most files: what a rule has to fill in, what a dialect has to fill in, and which of those places the suite checks for you. CONTRIBUTING.md is the same page for someone who has not cloned yet.
maak test (and maak emacs-test if the
package was touched).tests/checkers/, then run
maak self-check, maak check and maak fixture-check.While working, maak test-suite runs the one suite whose file is open, in a
second, instead of the three minutes the whole tree takes:
maak test-suite tests/checkers/tabs.scm
The project’s own style is the style it enforces, so it formats itself:
maak fmt # format sift's own sources with sift maak test # the Veritas suite maak test-suite # one suite, by file: tests/checkers/tabs.scm maak check # compile every module maak self-check # sift's sources are formatted by sift maak fixture-check # the golden corpus still round-trips maak docs # regenerate doc/rules.org maak emacs-check # byte-compile the package, warnings are errors maak emacs-test # ERT maak editors-check # Vim in batch, Node over the VS Code logic
Docstrings are Texinfo-flavoured (@var{}, @code{}), the first line is a
sentence, and each paragraph states the reason as well as the rule – which is
why this README can be read without a specification beside it.
A commit message is a type and then a sentence: feat: for something that did
not exist before, fix: for something that was wrong, and docs:, test:,
refactor:, build:, ci: or chore: for the rest, scoped in parentheses when
that narrows the subject (fix(lsp):, refactor(reader):). The reader who
matters is the one scanning git log for the change that broke a build, and the
type is what lets nine commits in ten be skipped unopened.
Patches arrive as a pull request on GitHub or as a patch sent with
git format-patch and git send-email, which is how the author usually works.
The review concerns the code.
Questions are equally welcome as issues. A question that the README does not answer is a defect in the README, and reporting it is the most useful bug report there is.
sift lsp answers, the ones it
refuses, and the code actions it offers.maak gates, the pipeline that runs them and the
ritual that cuts a release, what the benchmarks measure, and the module map.sift explain RULE — the same reasoning, one rule at a time, from the
program.sift list-checkers — the catalogue as a table, filtered by dialect.sift lsp implements, and
the source of the refusals it is allowed to make.sift is free software under the MIT license; LICENSE carries the text. A short permissive license is a practical requirement rather than a sentiment: a formatter that spreads through other people’s pipelines cannot carry a licensing question with it.
Texto de github.com tal como lo publicó, leído de su página y mostrado aquí con el formato de El Autómata. El original, en github.com · Volver a la noticia de El Autómata