remark-disable-tokenizers not working with remark 13, options?

[janpe]

remark-disable-tokenizers seems not to work with remark 13. Are there currently any options for reaching the same result?

I'd like to have

- List
- Of
- Items

to be output as text unchanged instead of making it a list.

[wooorm]

Looks like some of the markdown in your question is not working: See the CommonMark Help page for more on markdown.

As it’s a community plugin, it might be good to ask and/or solve the problem there? Notably, there are ways to turn off constructs indeed.

As your question is explained in only a few sentences, it might also be an XY problem. If you could spend more time framing your question, we might be able to offer a better solution.

[ChristianMurphy]

solution is either to influence the data that is coming to the client (which I cannot do)

You're using react-markdown, usually that is used in an authoring experience.
In which case, yes, you do have the ability to influence over what is coming from the client.

Reading between the lines a bit there may be legacy content which expects some markdown syntax to be treated as plain text.
That could be migrated to plain markdown, by creating a plugin, to escape constructs which previously rendered as plain text, see https://astexplorer.net/#/gist/1898fb07a2e8e6b80e5a8ba403b9dd52/3752ac9092b70528db993eedf19a45dd1f97ca47 and https://unifiedjs.com/learn then saving the content back to where it is stored, and using markdown going forward.

[janpe]

Thanks Christian. I hope you can respect the fact that I have to work with the constraints that I have described. In the long run a more robust system is possible but right here and now I need to manage the situation and I prefer using a plugin that works with the newer version rather than relying on an older version of react-markdown, which also similarly requires the remark-disable-tokenizers plugin to achieve what I need.

Thanks for providing some ideas, it has helped me figure out a way to handle the situation as it is right now.

[WCByrne]

@janpe Did you ever make any progress on this? I'm in the same situation but thought I'd check in before jumping in to trying to learn how to update the plugin.

[janpe]

@WCByrne I did manage to create a tiny plugin as per the comments on this issue: zestedesavoir/zmarkdown#416 (comment)

[WCByrne]

So I got a little "plugin" working but it seems like I can't disable some things. Notably images. Is there a definitive list of values that can be disabled? Found this list of constructs

Also, I need to support strikethrough which only appears to be part of GFM, but when I do that it seems I can't disable any of the other GFM additions like tables.

I feel like I'm starting to spin circles trying to navigate the documentation on all of this.

[ChristianMurphy]

Taking a step back for a moment.
Why are you looking for something markdown-like, instead of using markdown?
Where is the content coming from? (if not from react-markdown or remark, why is that system not generating the HTML?)
Context helps a lot in answering questions. 🙂

---

remark-disable-tokenizers seems not to work with remark 13. Are there currently any options for reaching the same result?

One possibility, would be to address zestedesavoir/zmarkdown#416, allowing the plugin to work with remark 13.

[samdenty]

for anyone looking for a solution to remark-disable-tokenizers in remark 13+:

import { SKIP, visit } from 'unist-util-visit';
import type { Plugin } from 'unified';
import type { UnistNode, UnistParent } from 'node_modules/unist-util-visit/lib';

const limitedMarkdownPlugin: Plugin = () => {
  return (tree, file) => {
    const contents = file.toString();

    visit(tree, (node: UnistNode, index, parent: UnistParent) => {
      if (
        index == null ||
        ['paragraph', 'text', 'inlineCode', 'code', 'strong', 'emphasis'].includes(node.type) ||
        !node.position
      ) {
        return true;
      }

      let value = contents.slice(node.position.start.offset, node.position.end.offset);

      if (node.type === 'heading') {
        value = `\n${value}`;
      }

      parent.children[index] = {
        type: 'text',
        value,
      } as any;

      return [SKIP, index] as const;
    });
  };
};

[wooorm]

That looks fragile. I a) recommend not doing this, b) recommend going with https://github.com/micromark/micromark#case-turn-off-constructs