Is there room for a plugin to apply expected values from certain vfile messages?

[tvquizphd]

For my particular application, there are some situations where I would like to automatically replace text in markdown files based on rules defined by retext plugins. One simple example would be "smart quotes". To illustrate this, I've created a CodeSandbox demo to replace smart quotes with dumb quotes. Here's a summary with many of the details commented out:

const sourceMarkdown = "They’re “smart” quotes!";

const remark_replace = () => {
  // a transformer to apply fixes from certain messages
  return (tree, file) => {
    const fixes = [];
    file.messages.forEach((message) => {
      // push relevant message offsets and expected values to fixes
      ...
      }
    });
    return unist_map(tree, (node) => {
      if (unist_is(node, "text")) {
        // extract node start and end offsets
        ...
        // filter fixes within start and end offsets
        const node_fixes = fixes.filter(...)
        ...
        // set node value with relevant fixes applied
        node.value = ...
      }
      return node;
    });
  };
}

const md_to_md_processor = unified()
  .use(remark_parse)
  .use(
    remark_retext,
    unified().use(retext_english).use(retext_quotes, { preferred: "straight" })
  )
  .use(remark_replace)
  .use(remark_stringify);

const { contents } = md_to_md_processor().processSync(sourceMarkdown);

I'm sorry if this question has already been raised or dismissed, but I've been searching Github for a few days and haven't found anyone who has been asking similar questions. I'd be happy to be pointed in that direction as well.

In theory, such a plugin could allow custom handlers for each VMessage source, providing the entire VMessage as input and returning a single string as output. Perhaps automatic replacement of text (without analysis of the surrounding context) only makes sense for a handful of plugins ("retext-quotes", "retext-contractions", "retext-diacritics", "retext-indefinite-article", "retext-profanities", "retext-repeated-words", "retext-sentence-spacing"). I think it would make sense for such a plugin to only support automatic replacement of a user-provided subset of VMessage sources.

I'd be happy to start work on such a plugin, unless I'm missing any issues that would render this project a non-starter.

[wooorm]

Hi there! I believe that should somewhat be possible yes!

Although, unified processors also change the AST and then serialize it.
But assuming that doesn’t happen, some retext plugins already come with actual and expected fields. And they have positional info. That info could be passed to for example magic-string.

Although, I’m a fan of ASTs. And this is string mutations. String mutations is what ESLint also does, and they’re a bit unhappy with them (a small typo and it could accidentally create invalid code). Which is why I’d prefer CSTs instead.

See also: remarkjs/remark-lint#82, retextjs/retext-repeated-words#3, retextjs/retext-readability#8, retextjs/retext-readability#8

[tvquizphd]

Awesome! Thanks for the response

Although, I’m a fan of ASTs. And this is string mutations. String mutations is what ESLint also does, and they’re a bit unhappy with them (a small typo and it could accidentally create invalid code). Which is why I’d prefer CSTs instead.

I'm currently working on a solution that parses each mdast text node into an nlcst. I'm comparing each node position to the positional info in the messages. Although it's probably overkill for my dataset, I hope I can address something along the lines of this example:

Input:

The link *is* is here [*here*](https://unified.com)

Output:

The link *is* [*here*](https://unified.com)

[tvquizphd]

I believe I've done it! Here's the plugin on CodeSandbox. I'm going with a tentative name of remark-autofix.

TLDR: Unless anyone has anything to add at this stage, I'll make a repository for remark-autofix and start writing some tests with various retext plugins.

My plugin applies messages to all Literal mdast nodes by modifying a filtered nlcst tree for each node. It applies "expected" values from the vfile messages for a variety of retext plugins. For now I've tested "retext-repeated-words", "retext-sentence-spacing", and "retext-spell".

Here's an example showing all three retext plugins working together.

This input markdown

Onne espace 'twen sentncos.  OK?
The link ia *ia* ia heeree [*heeree*](https://unified.com)

produces this output markdown

One space 'tween sentences. OK?
The link *is* [*here*](https://unified.com)

Here's how a user would apply the remark-autofix plugin in conjunction with remark-retext in bridge mode:

const md_to_md_processor = unified()
  .use(remark_parse)
  .use(
    remark_retext,
    unified()
      .use(retext_english)
      .use(retext_spacing)
      .use(retext_spell, dictionary)
      .use(retext_repeated)
  )
  .use(remark_autofix)
  .use(remark_stringify);

The plugin as written currently modifies the syntax tree in the Compiler phase instead of the Transformer phase. That was simply an arbitrary choice I made when I designed the plugin to pass a custom handler to remark-stringify.

To better integrate with the unified environment, I could easily rework the plugin to run during the Transformer phase.

[tvquizphd]

I've published a remark plugin to NPM that applies fixes to the MDAST during the Transforms stage. I'll be adding various tests and thinking of edge cases for a while, but I'll go ahead and mark this question as answered.

[wooorm]

Super interesting! Cool!