Add support for Frontmatter

[jlangeveld]

Plugin Version

v2.1.0

Prettier Version

v2.6.2

Which frameworks are affected?

• none
• vue
• angular
• svelte

Node Version

v18.3.0

Which operating systems have you used?

• Linux
• macOS
• Windows

Prettier config

{

}

Input

---
title: Hello world
---
doctype
html
  head
    title= title
  body
    h1 Hello world

Output or Error

- --
title: Hello world
- --
doctype
html
  head
    title= title
  body
    h1 Hello world

Expected Output

---
title: Hello world
---
doctype
html
  head
    title= title
  body
    h1 Hello world

Additional Context

This makes pug templates unusable in frameworks like eleventy, which parse and use the frontmatter keys.

[Shinigami92]

@jlangeveld Which file-extension does such a frontmatter file have?
Do you maybe have a repository where I can look into to reproduce this, so I can experiment with that to add support for frontmatter?

[jlangeveld]

The frontmatter is part of a normal pug-file (.pug extension)

The 11ty docs explain this much better than I do:
https://www.11ty.dev/docs/data-frontmatter/

[jlangeveld]

This is a bug, not a feature request..

[Shinigami92]

This is a bug, not a feature request..

Please note that I privately work in my free time on this project when I have time and I don't get any money for that!
You are always free to open a PR and try to tackle it yourself.
Also note that I have almost no experience with frontmatter and as far as I read through your send link, frontmatter has a yaml format at the top, so how in the world can pug handle yaml syntax out of nowhere 🤷
It is definitely not a but in my opinion but a feature request because you are asking for support of a whole new think that needs to be supported.

You are also very welcome to read a blogpost written by @antfu https://antfu.me/posts/why-reproductions-are-required

[jlangeveld]

Please note that I privately work in my free time on this project when I have time and I don't get any money for that!

I realise and respect that, apologies if I implied otherwise.

It is definitely not a bug in my opinion but a feature request because you are asking for support of a whole new think that needs to be supported.

I suppose you're right

Also note that I have almost no experience with frontmatter and as far as I read through your send link, frontmatter has a yaml format at the top, so how in the world can pug handle yaml syntax out of nowhere

While checking out implementations, I noticed that prettier has builtin support for this for the css and markdown languages. I will attempt implementing it, doesn't look too hard.

[Shinigami92]

Thank you for your understanding

While checking out implementations, I noticed that prettier has builtin support for this for the css and markdown languages. I will attempt implementing it, doesn't look too hard.

I also thought about something like this, but hadn't yet the time 🙁
We also could look if it's somehow possible to decide between yaml and json for the frontmatter part, because the docs say something about that's possible to configure via json.
On the other side, maybe we don't care about json for now as almost nobody uses it, I assume.
Like I have 12.5k weekly downloads with this plugin, 0.1% of them seems to use frontmatter 😉 = 13 persons, and maybe 0.01% of them uses json in frontmatter inside pug = 0 persons.

If you open a PR in draft mode, I hope I can help here and there, otherwise I cannot promise but maybe upcoming weekend I have more time. But currently it's a bit stressful for me.

[jlangeveld]

I also thought about something like this, but hadn't yet the time 🙁 We also could look if it's somehow possible to decide between yaml and json for the frontmatter part, because the docs say something about that's possible to configure via json.

Actually, that's taken care of by prettier's .../utils/front-matter implementation. All we have to do is hook that up at the beginning and end of the parse function and we have it working basically for free!

If you open a PR in draft mode, I hope I can help here and there

Never done that before, but there's a lot of docs, so that won't be a problem. I'll handle that later today or tomorrow.

otherwise I cannot promise but maybe upcoming weekend I have more time. But currently it's a bit stressful for me.

No problem, I'll prepare some code and testcases and set it up for review

[Shinigami92]

I hope it's as easy as that, but I hope you will not get afraid when you see how I implemented this plugin and that I don't use the prettier's plugin utilities 👀
I fully read and process in a token-stream (array) and not an AST, because pug works better that way and it safes performance

[jlangeveld]

I hope it's as easy as that, but I hope you will not get afraid when you see how I implemented this plugin and that I don't use the prettier's plugin utilities 👀 I fully read and process in a token-stream (array) and not an AST, because pug works better that way and it safes performance

Your printer class is initialised with content as 1st param, coming from the main plugin. Could I use that to stuff in the parsed front-matter from the parse?

[Shinigami92]

Your printer class is initialised with content as 1st param, coming from the main plugin. Could I use that to stuff in the parsed front-matter from the parse?

Uhm, sorry I don't fully understand your question (I'm not english native)
But try everything you want to support it, I have a really strong and big test-suite 😜
As long as all current tests are green + your newly added tests are also green, everything is welcome 🙌 (except hidden side-effects like bitcoin-miners 😜)

[Shinigami92]

Uhm... I found this 👀 https://www.npmjs.com/package/front-matter-pug

Also for me, it looks like that the frontmatter block at the top is not native pug 🤔

[jlangeveld]

Yeah thanks, I know about that one too. It's unmaintained...
At first I wanted to use the frontmatter stuff from prettier-core, but it's not available in plugins.
I used https://github.com/jonschlinkert/gray-matter, it's being used by 11ty too.

I have a working implementation, currently fixing testcases and working out json frontmatter.

[Shinigami92]

Okay... It could work.

We have two options I assume:

Option 1

When passing your example file into the tokenizer we get

Tokens

[
    {"type":"code","loc":{"start":{"line":1,"column":1},"end":{"line":1,"column":4}},"val":"--","mustEscape":false,"buffer":false},
    {"type":"newline","loc":{"start":{"line":2,"column":1},"end":{"line":2,"column":1}}},
    {"type":"tag","loc":{"start":{"line":2,"column":1},"end":{"line":2,"column":6}},"val":"title"},
    {"type":":","loc":{"start":{"line":2,"column":6},"end":{"line":2,"column":8}}},
    {"type":"tag","loc":{"start":{"line":2,"column":8},"end":{"line":2,"column":13}},"val":"Hello"},
    {"type":"text","loc":{"start":{"line":2,"column":14},"end":{"line":2,"column":19}},"val":"world"},
    {"type":"newline","loc":{"start":{"line":3,"column":1},"end":{"line":3,"column":1}}},
    {"type":"code","loc":{"start":{"line":3,"column":1},"end":{"line":3,"column":4}},"val":"--","mustEscape":false,"buffer":false},
    {"type":"newline","loc":{"start":{"line":4,"column":1},"end":{"line":4,"column":1}}},
    // ... vvv real pug starts below vvv
    {"type":"doctype","loc":{"start":{"line":4,"column":1},"end":{"line":4,"column":8}},"val":""},
    {"type":"newline","loc":{"start":{"line":5,"column":1},"end":{"line":5,"column":1}}},
    {"type":"tag","loc":{"start":{"line":5,"column":1},"end":{"line":5,"column":5}},"val":"html"},
    {"type":"indent","loc":{"start":{"line":6,"column":1},"end":{"line":6,"column":3}},"val":2},
    {"type":"tag","loc":{"start":{"line":6,"column":3},"end":{"line":6,"column":7}},"val":"head"},
    {"type":"indent","loc":{"start":{"line":7,"column":1},"end":{"line":7,"column":5}},"val":4},
    {"type":"tag","loc":{"start":{"line":7,"column":5},"end":{"line":7,"column":10}},"val":"title"},
    {"type":"code","loc":{"start":{"line":7,"column":10},"end":{"line":7,"column":17}},"val":"title","mustEscape":true,"buffer":true},
    {"type":"outdent","loc":{"start":{"line":8,"column":1},"end":{"line":8,"column":3}}},
    {"type":"tag","loc":{"start":{"line":8,"column":3},"end":{"line":8,"column":7}},"val":"body"},
    {"type":"indent","loc":{"start":{"line":9,"column":1},"end":{"line":9,"column":5}},"val":4},
    {"type":"tag","loc":{"start":{"line":9,"column":5},"end":{"line":9,"column":7}},"val":"h1"},
    {"type":"text","loc":{"start":{"line":9,"column":8},"end":{"line":9,"column":19}},"val":"Hello world"},
    {"type":"outdent","loc":{"start":{"line":10,"column":1},"end":{"line":10,"column":1}}},
    {"type":"outdent","loc":{"start":{"line":10,"column":1},"end":{"line":10,"column":1}}},
    {"type":"eos","loc":{"start":{"line":10,"column":1},"end":{"line":10,"column":1}}}
]

Screenshot of diff

When I remove the frontmatter yaml part, and let the document start with the doctype, we are kinda lucky as we got exactly the same result starting from the doctype tag but all lines are offset. We need to take this into account because some few things depend on the start.line and end.line.

So we could test for token.type === 'code' && token.val === '--' && token.mustEscape === false && token.buffer === false
You see we have 2 tokens with that "condition". You could try to concat the text inbetween and pass this somehow to prettier's yaml formatter.

Option 2

Instead of passing the whole file content into pug's tokenizer, we could test from the start of the file for these --- delimiters and handle the part separately.
Then we only pass the substring of the file content to pug's tokenizer.
As same as in option 1, we need to adjust the lines again in this case.