diff options
Diffstat (limited to 'packages/integrations/mdx/src')
-rw-r--r-- | packages/integrations/mdx/src/README.md | 124 | ||||
-rw-r--r-- | packages/integrations/mdx/src/index.ts | 151 | ||||
-rw-r--r-- | packages/integrations/mdx/src/plugins.ts | 99 | ||||
-rw-r--r-- | packages/integrations/mdx/src/rehype-apply-frontmatter-export.ts | 113 | ||||
-rw-r--r-- | packages/integrations/mdx/src/rehype-collect-headings.ts | 11 | ||||
-rw-r--r-- | packages/integrations/mdx/src/rehype-images-to-component.ts | 166 | ||||
-rw-r--r-- | packages/integrations/mdx/src/rehype-meta-string.ts | 17 | ||||
-rw-r--r-- | packages/integrations/mdx/src/rehype-optimize-static.ts | 302 | ||||
-rw-r--r-- | packages/integrations/mdx/src/server.ts | 73 | ||||
-rw-r--r-- | packages/integrations/mdx/src/utils.ts | 108 | ||||
-rw-r--r-- | packages/integrations/mdx/src/vite-plugin-mdx-postprocess.ts | 148 | ||||
-rw-r--r-- | packages/integrations/mdx/src/vite-plugin-mdx.ts | 106 |
12 files changed, 1418 insertions, 0 deletions
diff --git a/packages/integrations/mdx/src/README.md b/packages/integrations/mdx/src/README.md new file mode 100644 index 000000000..5c01ce755 --- /dev/null +++ b/packages/integrations/mdx/src/README.md @@ -0,0 +1,124 @@ +# Internal documentation + +## rehype-optimize-static + +The `rehype-optimize-static` plugin helps optimize the intermediate [`hast`](https://github.com/syntax-tree/hast) when processing MDX, collapsing static subtrees of the `hast` as a `"static string"` in the final JSX output. Here's a "before" and "after" result: + +Before: + +```jsx +function _createMdxContent() { + return ( + <> + <h1>My MDX Content</h1> + <pre> + <code class="language-js"> + <span class="token function">console</span> + <span class="token punctuation">.</span> + <span class="token function">log</span> + <span class="token punctuation">(</span> + <span class="token string">'hello world'</span> + <span class="token punctuation">)</span> + </code> + </pre> + </> + ); +} +``` + +After: + +```jsx +function _createMdxContent() { + return <Fragment set:html="<h1>My MDX Content</h1>\n<code class=...</code>" />; +} +``` + +> NOTE: If one of the nodes in `pre` is MDX, the optimization will not be applied to `pre`, but could be applied to the inner MDX node if its children are static. + +This results in fewer JSX nodes, less compiled JS output, and less parsed AST, which results in faster Rollup builds and runtime rendering. + +To achieve this, we use an algorithm to detect `hast` subtrees that are entirely static (containing no JSX) to be inlined as `set:html` to the root of the subtree. + +The next section explains the algorithm, which you can follow along by pairing with the [source code](./rehype-optimize-static.ts). To analyze the `hast`, you can paste the MDX code into https://mdxjs.com/playground. + +### How it works + +The flow can be divided into a "scan phase" and a "mutation phase". The scan phase searches for nodes that can be optimized, and the mutation phase applies the optimization on the `hast` nodes. + +#### Scan phase + +Variables: + +- `allPossibleElements`: A set of subtree roots where we can add a new `set:html` property with its children as value. +- `elementStack`: The stack of elements (that could be subtree roots) while traversing the `hast` (node ancestors). +- `elementMetadatas`: A weak map to store the metadata used only by the mutation phase later. + +Flow: + +1. Walk the `hast` tree. +2. For each `node` we enter, if the `node` is static (`type` is `element` or starts with `mdx`), record in `allPossibleElements` and push to `elementStack`. We also record additional metadata in `elementMetadatas` for the mutation phase later. + - Q: Why do we record `mdxJsxFlowElement`, it's MDX? <br> + A: Because we're looking for nodes whose children are static. The node itself doesn't need to be static. + - Q: Are we sure this is the subtree root node in `allPossibleElements`? <br> + A: No, but we'll clear that up later in step 3. +3. For each `node` we leave, pop from `elementStack`. If the `node`'s parent is in `allPossibleElements`, we also remove the `node` from `allPossibleElements`. + - Q: Why do we check for the node's parent? <br> + A: Checking for the node's parent allows us to identify a subtree root. When we enter a subtree like `C -> D -> E`, we leave in reverse: `E -> D -> C`. When we leave `E`, we see that it's parent `D` exists, so we remove `E`. When we leave `D`, we see `C` exists, so we remove `D`. When we leave `C`, we see that its parent doesn't exist, so we keep `C`, a subtree root. +4. _(Returning to the code written for step 2's `node` enter handling)_ We also need to handle the case where we find non-static elements. If found, we remove all the elements in `elementStack` from `allPossibleElements`. This happens before the code in step 2. + - Q: Why? <br> + A: Because if the `node` isn't static, that means all its ancestors (`elementStack`) have non-static children. So, the ancestors couldn't be a subtree root to be optimized anymore. + - Q: Why before step 2's `node` enter handling? <br> + A: If we find a non-static `node`, the `node` should still be considered in `allPossibleElements` as its children could be static. +5. Walk done. This leaves us with `allPossibleElements` containing only subtree roots that can be optimized. +6. Proceed to the mutation phase. + +#### Mutation phase + +Inputs: + +- `allPossibleElements` from the scan phase. +- `elementMetadatas` from the scan phase. + +Flow: + +1. Before we mutate the `hast` tree, each element in `allPossibleElements` may have siblings that can be optimized together. Sibling elements are grouped with the `findElementGroups()` function, which returns an array of element groups (new variable `elementGroups`) and mutates `allPossibleElements` to remove elements that are already part of a group. + + - Q: How does `findElementGroups()` work? <br> + A: For each elements in `allPossibleElements` that are non-static, we're able to take the element metadata from `elementMetadatas` and guess the next sibling node. If the next sibling node is static and is an element in `allPossibleElements`, we group them together for optimization. It continues to guess until it hits a non-static node or an element not in `allPossibleElements`, which it'll finalize the group as part of the returned result. + +2. For each elements in `allPossibleElements`, we serialize them as HTML and add it to the `set:html` property of the `hast` node, and remove its children. +3. For each element group in `elementGroups`, we serialize the group children as HTML and add it to a new `<Fragment set:html="..." />` node, and replace the group children with the new `<Fragment />` node. +4. 🎉 The rest of the MDX pipeline will do its thing and generate the desired JSX like above. + +### Extra + +#### MDX custom components + +Astro's MDX implementation supports specifying `export const components` in the MDX file to render some HTML elements as Astro components or framework components. `rehype-optimize-static` also needs to parse this JS to recognize some elements as non-static. + +#### Further optimizations + +In [Scan phase](#scan-phase) step 4, + +> we remove all the elements in `elementStack` from `allPossibleElements` + +We can further optimize this by then also emptying the `elementStack`. This ensures that if we run this same flow for a deeper node in the tree, we don't remove the already-removed nodes from `allPossibleElements`. + +While this breaks the concept of `elementStack`, it doesn't matter as the `elementStack` array pop in the "leave" handler (in step 3) would become a no-op. + +Example `elementStack` value during walking phase: + +``` +Enter: A +Enter: A, B +Enter: A, B, C +(Non-static node found): <empty> +Enter: D +Enter: D, E +Leave: D +Leave: <empty> +Leave: <empty> +Leave: <empty> +Leave: <empty> +``` diff --git a/packages/integrations/mdx/src/index.ts b/packages/integrations/mdx/src/index.ts new file mode 100644 index 000000000..fd2fab8c8 --- /dev/null +++ b/packages/integrations/mdx/src/index.ts @@ -0,0 +1,151 @@ +import fs from 'node:fs/promises'; +import { fileURLToPath } from 'node:url'; +import { markdownConfigDefaults } from '@astrojs/markdown-remark'; +import type { + AstroIntegration, + AstroIntegrationLogger, + ContainerRenderer, + ContentEntryType, + HookParameters, +} from 'astro'; +import type { Options as RemarkRehypeOptions } from 'remark-rehype'; +import type { PluggableList } from 'unified'; +import type { OptimizeOptions } from './rehype-optimize-static.js'; +import { ignoreStringPlugins, safeParseFrontmatter } from './utils.js'; +import { vitePluginMdxPostprocess } from './vite-plugin-mdx-postprocess.js'; +import { type VitePluginMdxOptions, vitePluginMdx } from './vite-plugin-mdx.js'; + +export type MdxOptions = Omit<typeof markdownConfigDefaults, 'remarkPlugins' | 'rehypePlugins'> & { + extendMarkdownConfig: boolean; + recmaPlugins: PluggableList; + // Markdown allows strings as remark and rehype plugins. + // This is not supported by the MDX compiler, so override types here. + remarkPlugins: PluggableList; + rehypePlugins: PluggableList; + remarkRehype: RemarkRehypeOptions; + optimize: boolean | OptimizeOptions; +}; + +type SetupHookParams = HookParameters<'astro:config:setup'> & { + // `addPageExtension` and `contentEntryType` are not a public APIs + // Add type defs here + addPageExtension: (extension: string) => void; + addContentEntryType: (contentEntryType: ContentEntryType) => void; +}; + +export function getContainerRenderer(): ContainerRenderer { + return { + name: 'astro:jsx', + serverEntrypoint: '@astrojs/mdx/server.js', + }; +} + +export default function mdx(partialMdxOptions: Partial<MdxOptions> = {}): AstroIntegration { + // @ts-expect-error Temporarily assign an empty object here, which will be re-assigned by the + // `astro:config:done` hook later. This is so that `vitePluginMdx` can get hold of a reference earlier. + let vitePluginMdxOptions: VitePluginMdxOptions = {}; + + return { + name: '@astrojs/mdx', + hooks: { + 'astro:config:setup': async (params) => { + const { updateConfig, config, addPageExtension, addContentEntryType, addRenderer } = + params as SetupHookParams; + + addRenderer({ + name: 'astro:jsx', + serverEntrypoint: new URL('../dist/server.js', import.meta.url), + }); + addPageExtension('.mdx'); + addContentEntryType({ + extensions: ['.mdx'], + async getEntryInfo({ fileUrl, contents }: { fileUrl: URL; contents: string }) { + const parsed = safeParseFrontmatter(contents, fileURLToPath(fileUrl)); + return { + data: parsed.frontmatter, + body: parsed.content.trim(), + slug: parsed.frontmatter.slug, + rawData: parsed.rawFrontmatter, + }; + }, + contentModuleTypes: await fs.readFile( + new URL('../template/content-module-types.d.ts', import.meta.url), + 'utf-8', + ), + // MDX can import scripts and styles, + // so wrap all MDX files with script / style propagation checks + handlePropagation: true, + }); + + updateConfig({ + vite: { + plugins: [vitePluginMdx(vitePluginMdxOptions), vitePluginMdxPostprocess(config)], + }, + }); + }, + 'astro:config:done': ({ config, logger }) => { + // We resolve the final MDX options here so that other integrations have a chance to modify + // `config.markdown` before we access it + const extendMarkdownConfig = + partialMdxOptions.extendMarkdownConfig ?? defaultMdxOptions.extendMarkdownConfig; + + const resolvedMdxOptions = applyDefaultOptions({ + options: partialMdxOptions, + defaults: markdownConfigToMdxOptions( + extendMarkdownConfig ? config.markdown : markdownConfigDefaults, + logger, + ), + }); + + // Mutate `mdxOptions` so that `vitePluginMdx` can reference the actual options + Object.assign(vitePluginMdxOptions, { + mdxOptions: resolvedMdxOptions, + srcDir: config.srcDir, + }); + // @ts-expect-error After we assign, we don't need to reference `mdxOptions` in this context anymore. + // Re-assign it so that the garbage can be collected later. + vitePluginMdxOptions = {}; + }, + }, + }; +} + +const defaultMdxOptions = { + extendMarkdownConfig: true, + recmaPlugins: [], + optimize: false, +} satisfies Partial<MdxOptions>; + +function markdownConfigToMdxOptions( + markdownConfig: typeof markdownConfigDefaults, + logger: AstroIntegrationLogger, +): MdxOptions { + return { + ...defaultMdxOptions, + ...markdownConfig, + remarkPlugins: ignoreStringPlugins(markdownConfig.remarkPlugins, logger), + rehypePlugins: ignoreStringPlugins(markdownConfig.rehypePlugins, logger), + remarkRehype: (markdownConfig.remarkRehype as any) ?? {}, + }; +} + +function applyDefaultOptions({ + options, + defaults, +}: { + options: Partial<MdxOptions>; + defaults: MdxOptions; +}): MdxOptions { + return { + syntaxHighlight: options.syntaxHighlight ?? defaults.syntaxHighlight, + extendMarkdownConfig: options.extendMarkdownConfig ?? defaults.extendMarkdownConfig, + recmaPlugins: options.recmaPlugins ?? defaults.recmaPlugins, + remarkRehype: options.remarkRehype ?? defaults.remarkRehype, + gfm: options.gfm ?? defaults.gfm, + smartypants: options.smartypants ?? defaults.smartypants, + remarkPlugins: options.remarkPlugins ?? defaults.remarkPlugins, + rehypePlugins: options.rehypePlugins ?? defaults.rehypePlugins, + shikiConfig: options.shikiConfig ?? defaults.shikiConfig, + optimize: options.optimize ?? defaults.optimize, + }; +} diff --git a/packages/integrations/mdx/src/plugins.ts b/packages/integrations/mdx/src/plugins.ts new file mode 100644 index 000000000..77c76243c --- /dev/null +++ b/packages/integrations/mdx/src/plugins.ts @@ -0,0 +1,99 @@ +import { + rehypeHeadingIds, + rehypePrism, + rehypeShiki, + remarkCollectImages, +} from '@astrojs/markdown-remark'; +import { createProcessor, nodeTypes } from '@mdx-js/mdx'; +import { rehypeAnalyzeAstroMetadata } from 'astro/jsx/rehype.js'; +import rehypeRaw from 'rehype-raw'; +import remarkGfm from 'remark-gfm'; +import remarkSmartypants from 'remark-smartypants'; +import { SourceMapGenerator } from 'source-map'; +import type { PluggableList } from 'unified'; +import type { MdxOptions } from './index.js'; +import { rehypeApplyFrontmatterExport } from './rehype-apply-frontmatter-export.js'; +import { rehypeInjectHeadingsExport } from './rehype-collect-headings.js'; +import { rehypeImageToComponent } from './rehype-images-to-component.js'; +import rehypeMetaString from './rehype-meta-string.js'; +import { rehypeOptimizeStatic } from './rehype-optimize-static.js'; + +// Skip nonessential plugins during performance benchmark runs +const isPerformanceBenchmark = Boolean(process.env.ASTRO_PERFORMANCE_BENCHMARK); + +interface MdxProcessorExtraOptions { + sourcemap: boolean; +} + +export function createMdxProcessor(mdxOptions: MdxOptions, extraOptions: MdxProcessorExtraOptions) { + return createProcessor({ + remarkPlugins: getRemarkPlugins(mdxOptions), + rehypePlugins: getRehypePlugins(mdxOptions), + recmaPlugins: mdxOptions.recmaPlugins, + remarkRehypeOptions: mdxOptions.remarkRehype, + jsxImportSource: 'astro', + // Note: disable `.md` (and other alternative extensions for markdown files like `.markdown`) support + format: 'mdx', + mdExtensions: [], + elementAttributeNameCase: 'html', + SourceMapGenerator: extraOptions.sourcemap ? SourceMapGenerator : undefined, + }); +} + +function getRemarkPlugins(mdxOptions: MdxOptions): PluggableList { + let remarkPlugins: PluggableList = []; + + if (!isPerformanceBenchmark) { + if (mdxOptions.gfm) { + remarkPlugins.push(remarkGfm); + } + if (mdxOptions.smartypants) { + remarkPlugins.push(remarkSmartypants); + } + } + + remarkPlugins.push(...mdxOptions.remarkPlugins, remarkCollectImages); + + return remarkPlugins; +} + +function getRehypePlugins(mdxOptions: MdxOptions): PluggableList { + let rehypePlugins: PluggableList = [ + // ensure `data.meta` is preserved in `properties.metastring` for rehype syntax highlighters + rehypeMetaString, + // rehypeRaw allows custom syntax highlighters to work without added config + [rehypeRaw, { passThrough: nodeTypes }], + ]; + + if (!isPerformanceBenchmark) { + // Apply syntax highlighters after user plugins to match `markdown/remark` behavior + if (mdxOptions.syntaxHighlight === 'shiki') { + rehypePlugins.push([rehypeShiki, mdxOptions.shikiConfig]); + } else if (mdxOptions.syntaxHighlight === 'prism') { + rehypePlugins.push(rehypePrism); + } + } + + rehypePlugins.push(...mdxOptions.rehypePlugins, rehypeImageToComponent); + + if (!isPerformanceBenchmark) { + // getHeadings() is guaranteed by TS, so this must be included. + // We run `rehypeHeadingIds` _last_ to respect any custom IDs set by user plugins. + rehypePlugins.push(rehypeHeadingIds, rehypeInjectHeadingsExport); + } + + rehypePlugins.push( + // Render info from `vfile.data.astro.frontmatter` as JS + rehypeApplyFrontmatterExport, + // Analyze MDX nodes and attach to `vfile.data.__astroMetadata` + rehypeAnalyzeAstroMetadata, + ); + + if (mdxOptions.optimize) { + // Convert user `optimize` option to compatible `rehypeOptimizeStatic` option + const options = mdxOptions.optimize === true ? undefined : mdxOptions.optimize; + rehypePlugins.push([rehypeOptimizeStatic, options]); + } + + return rehypePlugins; +} diff --git a/packages/integrations/mdx/src/rehype-apply-frontmatter-export.ts b/packages/integrations/mdx/src/rehype-apply-frontmatter-export.ts new file mode 100644 index 000000000..5880c30b3 --- /dev/null +++ b/packages/integrations/mdx/src/rehype-apply-frontmatter-export.ts @@ -0,0 +1,113 @@ +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { isFrontmatterValid } from '@astrojs/markdown-remark'; +import type { Root, RootContent } from 'hast'; +import type { VFile } from 'vfile'; +import { jsToTreeNode } from './utils.js'; + +// Passed metadata to help determine adding charset utf8 by default +declare module 'vfile' { + interface DataMap { + applyFrontmatterExport?: { + srcDir?: URL; + }; + } +} + +const exportConstPartialTrueRe = /export\s+const\s+partial\s*=\s*true/; + +export function rehypeApplyFrontmatterExport() { + return function (tree: Root, vfile: VFile) { + const frontmatter = vfile.data.astro?.frontmatter; + if (!frontmatter || !isFrontmatterValid(frontmatter)) + throw new Error( + // Copied from Astro core `errors-data` + // TODO: find way to import error data from core + '[MDX] A remark or rehype plugin attempted to inject invalid frontmatter. Ensure "astro.frontmatter" is set to a valid JSON object that is not `null` or `undefined`.', + ); + const extraChildren: RootContent[] = [ + jsToTreeNode(`export const frontmatter = ${JSON.stringify(frontmatter)};`), + ]; + if (frontmatter.layout) { + extraChildren.unshift( + jsToTreeNode( + // NOTE: Use `__astro_*` import names to prevent conflicts with user code + /** @see 'vite-plugin-markdown' for layout props reference */ + `\ +import { jsx as __astro_layout_jsx__ } from 'astro/jsx-runtime'; +import __astro_layout_component__ from ${JSON.stringify(frontmatter.layout)}; + +export default function ({ children }) { + const { layout, ...content } = frontmatter; + content.file = file; + content.url = url; + return __astro_layout_jsx__(__astro_layout_component__, { + file, + url, + content, + frontmatter: content, + headings: getHeadings(), + 'server:root': true, + children, + }); +};`, + ), + ); + } else if (shouldAddCharset(tree, vfile)) { + extraChildren.unshift({ + type: 'mdxJsxFlowElement', + name: 'meta', + attributes: [ + { + type: 'mdxJsxAttribute', + name: 'charset', + value: 'utf-8', + }, + ], + children: [], + }); + } + tree.children = extraChildren.concat(tree.children); + }; +} + +/** + * If this is a page (e.g. in src/pages), has no layout frontmatter (handled before calling this function), + * has no leading component that looks like a wrapping layout, and `partial` isn't set to true, we default to + * adding charset=utf-8 like markdown so that users don't have to worry about it for MDX pages without layouts. + */ +function shouldAddCharset(tree: Root, vfile: VFile) { + const srcDirUrl = vfile.data.applyFrontmatterExport?.srcDir; + if (!srcDirUrl) return false; + + const hasConstPartialTrue = tree.children.some( + (node) => node.type === 'mdxjsEsm' && exportConstPartialTrueRe.test(node.value), + ); + if (hasConstPartialTrue) return false; + + // NOTE: the pages directory is a non-configurable Astro behaviour + const pagesDir = path.join(fileURLToPath(srcDirUrl), 'pages').replace(/\\/g, '/'); + // `vfile.path` comes from Vite, which is a normalized path (no backslashes) + const filePath = vfile.path; + if (!filePath.startsWith(pagesDir)) return false; + + const hasLeadingUnderscoreInPath = filePath + .slice(pagesDir.length) + .replace(/\\/g, '/') + .split('/') + .some((part) => part.startsWith('_')); + if (hasLeadingUnderscoreInPath) return false; + + // Bail if the first content found is a wrapping layout component + for (const child of tree.children) { + if (child.type === 'element') break; + if (child.type === 'mdxJsxFlowElement') { + // If is fragment or lowercase tag name (html tags), skip and assume there's no layout + if (child.name == null) break; + if (child.name[0] === child.name[0].toLowerCase()) break; + return false; + } + } + + return true; +} diff --git a/packages/integrations/mdx/src/rehype-collect-headings.ts b/packages/integrations/mdx/src/rehype-collect-headings.ts new file mode 100644 index 000000000..a51e8e9f0 --- /dev/null +++ b/packages/integrations/mdx/src/rehype-collect-headings.ts @@ -0,0 +1,11 @@ +import type { VFile } from 'vfile'; +import { jsToTreeNode } from './utils.js'; + +export function rehypeInjectHeadingsExport() { + return function (tree: any, file: VFile) { + const headings = file.data.astro?.headings ?? []; + tree.children.unshift( + jsToTreeNode(`export function getHeadings() { return ${JSON.stringify(headings)} }`), + ); + }; +} diff --git a/packages/integrations/mdx/src/rehype-images-to-component.ts b/packages/integrations/mdx/src/rehype-images-to-component.ts new file mode 100644 index 000000000..c903ae511 --- /dev/null +++ b/packages/integrations/mdx/src/rehype-images-to-component.ts @@ -0,0 +1,166 @@ +import type { Properties, Root } from 'hast'; +import type { MdxJsxAttribute, MdxjsEsm } from 'mdast-util-mdx'; +import type { MdxJsxFlowElementHast } from 'mdast-util-mdx-jsx'; +import { visit } from 'unist-util-visit'; +import type { VFile } from 'vfile'; +import { jsToTreeNode } from './utils.js'; + +export const ASTRO_IMAGE_ELEMENT = 'astro-image'; +export const ASTRO_IMAGE_IMPORT = '__AstroImage__'; +export const USES_ASTRO_IMAGE_FLAG = '__usesAstroImage'; + +function createArrayAttribute(name: string, values: (string | number)[]): MdxJsxAttribute { + return { + type: 'mdxJsxAttribute', + name: name, + value: { + type: 'mdxJsxAttributeValueExpression', + value: name, + data: { + estree: { + type: 'Program', + body: [ + { + type: 'ExpressionStatement', + expression: { + type: 'ArrayExpression', + elements: values.map((value) => ({ + type: 'Literal', + value: value, + raw: String(value), + })), + }, + }, + ], + sourceType: 'module', + comments: [], + }, + }, + }, + }; +} + +/** + * Convert the <img /> element properties (except `src`) to MDX JSX attributes. + * + * @param {Properties} props - The element properties + * @returns {MdxJsxAttribute[]} The MDX attributes + */ +function getImageComponentAttributes(props: Properties): MdxJsxAttribute[] { + const attrs: MdxJsxAttribute[] = []; + + for (const [prop, value] of Object.entries(props)) { + if (prop === 'src') continue; + + /* + * <Image /> component expects an array for those attributes but the + * received properties are sanitized as strings. So we need to convert them + * back to an array. + */ + if (prop === 'widths' || prop === 'densities') { + attrs.push(createArrayAttribute(prop, String(value).split(' '))); + } else { + attrs.push({ + name: prop, + type: 'mdxJsxAttribute', + value: String(value), + }); + } + } + + return attrs; +} + +export function rehypeImageToComponent() { + return function (tree: Root, file: VFile) { + if (!file.data.astro?.imagePaths?.length) return; + const importsStatements: MdxjsEsm[] = []; + const importedImages = new Map<string, string>(); + + visit(tree, 'element', (node, index, parent) => { + if (!file.data.astro?.imagePaths?.length || node.tagName !== 'img' || !node.properties.src) + return; + + const src = decodeURI(String(node.properties.src)); + + if (!file.data.astro.imagePaths?.includes(src)) return; + + let importName = importedImages.get(src); + + if (!importName) { + importName = `__${importedImages.size}_${src.replace(/\W/g, '_')}__`; + + importsStatements.push({ + type: 'mdxjsEsm', + value: '', + data: { + estree: { + type: 'Program', + sourceType: 'module', + body: [ + { + type: 'ImportDeclaration', + source: { + type: 'Literal', + value: src, + raw: JSON.stringify(src), + }, + specifiers: [ + { + type: 'ImportDefaultSpecifier', + local: { type: 'Identifier', name: importName }, + }, + ], + }, + ], + }, + }, + }); + importedImages.set(src, importName); + } + + // Build a component that's equivalent to <Image src={importName} {...attributes} /> + const componentElement: MdxJsxFlowElementHast = { + name: ASTRO_IMAGE_ELEMENT, + type: 'mdxJsxFlowElement', + attributes: [ + ...getImageComponentAttributes(node.properties), + { + name: 'src', + type: 'mdxJsxAttribute', + value: { + type: 'mdxJsxAttributeValueExpression', + value: importName, + data: { + estree: { + type: 'Program', + sourceType: 'module', + comments: [], + body: [ + { + type: 'ExpressionStatement', + expression: { type: 'Identifier', name: importName }, + }, + ], + }, + }, + }, + }, + ], + children: [], + }; + + parent!.children.splice(index!, 1, componentElement); + }); + + // Add all the import statements to the top of the file for the images + tree.children.unshift(...importsStatements); + + tree.children.unshift( + jsToTreeNode(`import { Image as ${ASTRO_IMAGE_IMPORT} } from "astro:assets";`), + ); + // Export `__usesAstroImage` to pick up `astro:assets` usage in the module graph. + // @see the '@astrojs/mdx-postprocess' plugin + tree.children.push(jsToTreeNode(`export const ${USES_ASTRO_IMAGE_FLAG} = true`)); + }; +} diff --git a/packages/integrations/mdx/src/rehype-meta-string.ts b/packages/integrations/mdx/src/rehype-meta-string.ts new file mode 100644 index 000000000..c3f2dbd2f --- /dev/null +++ b/packages/integrations/mdx/src/rehype-meta-string.ts @@ -0,0 +1,17 @@ +import { visit } from 'unist-util-visit'; + +/** + * Moves `data.meta` to `properties.metastring` for the `code` element node + * as `rehype-raw` strips `data` from all nodes, which may contain useful information. + * e.g. ```js {1:3} => metastring: "{1:3}" + */ +export default function rehypeMetaString() { + return function (tree: any) { + visit(tree, (node) => { + if (node.type === 'element' && node.tagName === 'code' && node.data?.meta) { + node.properties ??= {}; + node.properties.metastring = node.data.meta; + } + }); + }; +} diff --git a/packages/integrations/mdx/src/rehype-optimize-static.ts b/packages/integrations/mdx/src/rehype-optimize-static.ts new file mode 100644 index 000000000..eba31cae0 --- /dev/null +++ b/packages/integrations/mdx/src/rehype-optimize-static.ts @@ -0,0 +1,302 @@ +import type { RehypePlugin } from '@astrojs/markdown-remark'; +import { SKIP, visit } from 'estree-util-visit'; +import type { Element, RootContent, RootContentMap } from 'hast'; +import { toHtml } from 'hast-util-to-html'; +import type { MdxJsxFlowElementHast, MdxJsxTextElementHast } from 'mdast-util-mdx-jsx'; + +// This import includes ambient types for hast to include mdx nodes +import type {} from 'mdast-util-mdx'; + +// Alias as the main hast node +type Node = RootContent; +// Nodes that have the `children` property +type ParentNode = Element | MdxJsxFlowElementHast | MdxJsxTextElementHast; +// Nodes that can have its children optimized as a single HTML string +type OptimizableNode = Element | MdxJsxFlowElementHast | MdxJsxTextElementHast; + +export interface OptimizeOptions { + ignoreElementNames?: string[]; +} + +interface ElementMetadata { + parent: ParentNode; + index: number; +} + +const exportConstComponentsRe = /export\s+const\s+components\s*=/; + +/** + * For MDX only, collapse static subtrees of the hast into `set:html`. Subtrees + * do not include any MDX elements. + * + * This optimization reduces the JS output as more content are represented as a + * string instead, which also reduces the AST size that Rollup holds in memory. + */ +export const rehypeOptimizeStatic: RehypePlugin<[OptimizeOptions?]> = (options) => { + return (tree) => { + // A set of non-static components to avoid collapsing when walking the tree + // as they need to be preserved as JSX to be rendered dynamically. + const ignoreElementNames = new Set<string>(options?.ignoreElementNames); + + // Find `export const components = { ... }` and get it's object's keys to be + // populated into `ignoreElementNames`. This configuration is used to render + // some HTML elements as custom components, and we also want to avoid collapsing them. + for (const child of tree.children) { + if (child.type === 'mdxjsEsm' && exportConstComponentsRe.test(child.value)) { + const keys = getExportConstComponentObjectKeys(child); + if (keys) { + for (const key of keys) { + ignoreElementNames.add(key); + } + } + break; + } + } + + // All possible elements that could be the root of a subtree + const allPossibleElements = new Set<OptimizableNode>(); + // The current collapsible element stack while traversing the tree + const elementStack: Node[] = []; + // Metadata used by `findElementGroups` later + const elementMetadatas = new WeakMap<OptimizableNode, ElementMetadata>(); + + /** + * A non-static node causes all its parents to be non-optimizable + */ + const isNodeNonStatic = (node: Node) => { + return ( + node.type.startsWith('mdx') || + // @ts-expect-error `node` should never have `type: 'root'`, but in some cases plugins may inject it as children, + // which MDX will render as a fragment instead (an MDX fragment is a `mdxJsxFlowElement` type). + node.type === 'root' || + // @ts-expect-error Access `.tagName` naively for perf + ignoreElementNames.has(node.tagName) + ); + }; + + visit(tree as any, { + // @ts-expect-error Force coerce node as hast node + enter(node: Node, key, index, parents: ParentNode[]) { + // `estree-util-visit` may traverse in MDX `attributes`, we don't want that. Only continue + // if it's traversing the root, or the `children` key. + if (key != null && key !== 'children') return SKIP; + + // Mutate `node` as a normal hast element node if it's a plain MDX node, e.g. `<kbd>something</kbd>` + simplifyPlainMdxComponentNode(node, ignoreElementNames); + + // For nodes that are not static, eliminate all elements in the `elementStack` from the + // `allPossibleElements` set. + if (isNodeNonStatic(node)) { + for (const el of elementStack) { + allPossibleElements.delete(el as OptimizableNode); + } + // Micro-optimization: While this destroys the meaning of an element + // stack for this node, things will still work but we won't repeatedly + // run the above for other nodes anymore. If this is confusing, you can + // comment out the code below when reading. + elementStack.length = 0; + } + // For possible subtree root nodes, record them in `elementStack` and + // `allPossibleElements` to be used in the "leave" hook below. + if (node.type === 'element' || isMdxComponentNode(node)) { + elementStack.push(node); + allPossibleElements.add(node); + + if (index != null && node.type === 'element') { + // Record metadata for element node to be used for grouping analysis later + elementMetadatas.set(node, { parent: parents[parents.length - 1], index }); + } + } + }, + // @ts-expect-error Force coerce node as hast node + leave(node: Node, key, _, parents: ParentNode[]) { + // `estree-util-visit` may traverse in MDX `attributes`, we don't want that. Only continue + // if it's traversing the root, or the `children` key. + if (key != null && key !== 'children') return SKIP; + + // Do the reverse of the if condition above, popping the `elementStack`, + // and consolidating `allPossibleElements` as a subtree root. + if (node.type === 'element' || isMdxComponentNode(node)) { + elementStack.pop(); + // Many possible elements could be part of a subtree, in order to find + // the root, we check the parent of the element we're popping. If the + // parent exists in `allPossibleElements`, then we're definitely not + // the root, so remove ourselves. This will work retroactively as we + // climb back up the tree. + const parent = parents[parents.length - 1]; + if (allPossibleElements.has(parent)) { + allPossibleElements.delete(node); + } + } + }, + }); + + // Within `allPossibleElements`, element nodes are often siblings and instead of setting `set:html` + // on each of the element node, we can create a `<Fragment set:html="...">` element that includes + // all element nodes instead, simplifying the output. + const elementGroups = findElementGroups(allPossibleElements, elementMetadatas, isNodeNonStatic); + + // For all possible subtree roots, collapse them into `set:html` and + // strip of their children + for (const el of allPossibleElements) { + // Avoid adding empty `set:html` attributes if there's no children + if (el.children.length === 0) continue; + + if (isMdxComponentNode(el)) { + el.attributes.push({ + type: 'mdxJsxAttribute', + name: 'set:html', + value: toHtml(el.children), + }); + } else { + el.properties['set:html'] = toHtml(el.children); + } + el.children = []; + } + + // For each element group, we create a new `<Fragment />` MDX node with `set:html` of the children + // serialized as HTML. We insert this new fragment, replacing all the group children nodes. + // We iterate in reverse to avoid changing the index of groups of the same parent. + for (let i = elementGroups.length - 1; i >= 0; i--) { + const group = elementGroups[i]; + const fragmentNode: MdxJsxFlowElementHast = { + type: 'mdxJsxFlowElement', + name: 'Fragment', + attributes: [ + { + type: 'mdxJsxAttribute', + name: 'set:html', + value: toHtml(group.children), + }, + ], + children: [], + }; + group.parent.children.splice(group.startIndex, group.children.length, fragmentNode); + } + }; +}; + +interface ElementGroup { + parent: ParentNode; + startIndex: number; + children: Node[]; +} + +/** + * Iterate through `allPossibleElements` and find elements that are siblings, and return them. `allPossibleElements` + * will be mutated to exclude these grouped elements. + */ +function findElementGroups( + allPossibleElements: Set<OptimizableNode>, + elementMetadatas: WeakMap<OptimizableNode, ElementMetadata>, + isNodeNonStatic: (node: Node) => boolean, +): ElementGroup[] { + const elementGroups: ElementGroup[] = []; + + for (const el of allPossibleElements) { + // Non-static nodes can't be grouped. It can only optimize its static children. + if (isNodeNonStatic(el)) continue; + + // Get the metadata for the element node, this should always exist + const metadata = elementMetadatas.get(el); + if (!metadata) { + throw new Error( + 'Internal MDX error: rehype-optimize-static should have metadata for element node', + ); + } + + // For this element, iterate through the next siblings and add them to this array + // if they are text nodes or elements that are in `allPossibleElements` (optimizable). + // If one of the next siblings don't match the criteria, break the loop as others are no longer siblings. + const groupableElements: Node[] = [el]; + for (let i = metadata.index + 1; i < metadata.parent.children.length; i++) { + const node = metadata.parent.children[i]; + + // If the node is non-static, we can't group it with the current element + if (isNodeNonStatic(node)) break; + + if (node.type === 'element') { + // This node is now (presumably) part of a group, remove it from `allPossibleElements` + const existed = allPossibleElements.delete(node); + // If this node didn't exist in `allPossibleElements`, it's likely that one of its children + // are non-static, hence this node can also not be grouped. So we break out here. + if (!existed) break; + } + + groupableElements.push(node); + } + + // If group elements are more than one, add them to the `elementGroups`. + // Grouping is most effective if there's multiple elements in it. + if (groupableElements.length > 1) { + elementGroups.push({ + parent: metadata.parent, + startIndex: metadata.index, + children: groupableElements, + }); + // The `el` is also now part of a group, remove it from `allPossibleElements` + allPossibleElements.delete(el); + } + } + + return elementGroups; +} + +function isMdxComponentNode(node: Node): node is MdxJsxFlowElementHast | MdxJsxTextElementHast { + return node.type === 'mdxJsxFlowElement' || node.type === 'mdxJsxTextElement'; +} + +/** + * Get the object keys from `export const components` + * + * @example + * `export const components = { foo, bar: Baz }`, returns `['foo', 'bar']` + */ +function getExportConstComponentObjectKeys(node: RootContentMap['mdxjsEsm']) { + const exportNamedDeclaration = node.data?.estree?.body[0]; + if (exportNamedDeclaration?.type !== 'ExportNamedDeclaration') return; + + const variableDeclaration = exportNamedDeclaration.declaration; + if (variableDeclaration?.type !== 'VariableDeclaration') return; + + const variableInit = variableDeclaration.declarations[0]?.init; + if (variableInit?.type !== 'ObjectExpression') return; + + const keys: string[] = []; + for (const propertyNode of variableInit.properties) { + if (propertyNode.type === 'Property' && propertyNode.key.type === 'Identifier') { + keys.push(propertyNode.key.name); + } + } + return keys; +} + +/** + * Some MDX nodes are simply `<kbd>something</kbd>` which isn't needed to be completely treated + * as an MDX node. This function tries to mutate this node as a simple hast element node if so. + */ +function simplifyPlainMdxComponentNode(node: Node, ignoreElementNames: Set<string>) { + if ( + !isMdxComponentNode(node) || + // Attributes could be dynamic, so bail if so. + node.attributes.length > 0 || + // Fragments are also dynamic + !node.name || + // Ignore if the node name is in the ignore list + ignoreElementNames.has(node.name) || + // If the node name has uppercase characters, it's likely an actual MDX component + node.name.toLowerCase() !== node.name + ) { + return; + } + + // Mutate as hast element node + const newNode = node as unknown as Element; + newNode.type = 'element'; + newNode.tagName = node.name; + newNode.properties = {}; + + // @ts-expect-error Delete mdx-specific properties + node.attributes = undefined; + node.data = undefined; +} diff --git a/packages/integrations/mdx/src/server.ts b/packages/integrations/mdx/src/server.ts new file mode 100644 index 000000000..79934eb32 --- /dev/null +++ b/packages/integrations/mdx/src/server.ts @@ -0,0 +1,73 @@ +import type { NamedSSRLoadedRendererValue } from 'astro'; +import { AstroError } from 'astro/errors'; +import { AstroJSX, jsx } from 'astro/jsx-runtime'; +import { renderJSX } from 'astro/runtime/server/index.js'; + +const slotName = (str: string) => str.trim().replace(/[-_]([a-z])/g, (_, w) => w.toUpperCase()); + +// NOTE: In practice, MDX components are always tagged with `__astro_tag_component__`, so the right renderer +// is used directly, and this check is not often used to return true. +export async function check( + Component: any, + props: any, + { default: children = null, ...slotted } = {}, +) { + if (typeof Component !== 'function') return false; + const slots: Record<string, any> = {}; + for (const [key, value] of Object.entries(slotted)) { + const name = slotName(key); + slots[name] = value; + } + try { + const result = await Component({ ...props, ...slots, children }); + return result[AstroJSX]; + } catch (e) { + throwEnhancedErrorIfMdxComponent(e as Error, Component); + } + return false; +} + +export async function renderToStaticMarkup( + this: any, + Component: any, + props = {}, + { default: children = null, ...slotted } = {}, +) { + const slots: Record<string, any> = {}; + for (const [key, value] of Object.entries(slotted)) { + const name = slotName(key); + slots[name] = value; + } + + const { result } = this; + try { + const html = await renderJSX(result, jsx(Component, { ...props, ...slots, children })); + return { html }; + } catch (e) { + throwEnhancedErrorIfMdxComponent(e as Error, Component); + throw e; + } +} + +function throwEnhancedErrorIfMdxComponent(error: Error, Component: any) { + // if the exception is from an mdx component + // throw an error + if (Component[Symbol.for('mdx-component')]) { + // if it's an existing AstroError, we don't need to re-throw, keep the original hint + if (AstroError.is(error)) return; + // Mimic the fields of the internal `AstroError` class (not from `astro/errors`) to + // provide better title and hint for the error overlay + (error as any).title = error.name; + (error as any).hint = + `This issue often occurs when your MDX component encounters runtime errors.`; + throw error; + } +} + +const renderer: NamedSSRLoadedRendererValue = { + name: 'astro:jsx', + check, + renderToStaticMarkup, +}; + +export default renderer; diff --git a/packages/integrations/mdx/src/utils.ts b/packages/integrations/mdx/src/utils.ts new file mode 100644 index 000000000..7dcd4a14c --- /dev/null +++ b/packages/integrations/mdx/src/utils.ts @@ -0,0 +1,108 @@ +import { parseFrontmatter } from '@astrojs/markdown-remark'; +import type { Options as AcornOpts } from 'acorn'; +import { parse } from 'acorn'; +import type { AstroConfig, AstroIntegrationLogger, SSRError } from 'astro'; +import { bold } from 'kleur/colors'; +import type { MdxjsEsm } from 'mdast-util-mdx'; +import type { PluggableList } from 'unified'; + +function appendForwardSlash(path: string) { + return path.endsWith('/') ? path : path + '/'; +} + +export interface FileInfo { + fileId: string; + fileUrl: string; +} + +/** @see 'vite-plugin-utils' for source */ +export function getFileInfo(id: string, config: AstroConfig): FileInfo { + const sitePathname = appendForwardSlash( + config.site ? new URL(config.base, config.site).pathname : config.base, + ); + + // Try to grab the file's actual URL + let url: URL | undefined = undefined; + try { + url = new URL(`file://${id}`); + } catch {} + + const fileId = id.split('?')[0]; + let fileUrl: string; + const isPage = fileId.includes('/pages/'); + if (isPage) { + fileUrl = fileId.replace(/^.*?\/pages\//, sitePathname).replace(/(?:\/index)?\.mdx$/, ''); + } else if (url?.pathname.startsWith(config.root.pathname)) { + fileUrl = url.pathname.slice(config.root.pathname.length); + } else { + fileUrl = fileId; + } + + if (fileUrl && config.trailingSlash === 'always') { + fileUrl = appendForwardSlash(fileUrl); + } + return { fileId, fileUrl }; +} + +/** + * Match YAML exception handling from Astro core errors + * @see 'astro/src/core/errors.ts' + */ +export function safeParseFrontmatter(code: string, id: string) { + try { + return parseFrontmatter(code, { frontmatter: 'empty-with-spaces' }); + } catch (e: any) { + if (e.name === 'YAMLException') { + const err: SSRError = e; + err.id = id; + err.loc = { file: e.id, line: e.mark.line + 1, column: e.mark.column }; + err.message = e.reason; + throw err; + } else { + throw e; + } + } +} + +export function jsToTreeNode( + jsString: string, + acornOpts: AcornOpts = { + ecmaVersion: 'latest', + sourceType: 'module', + }, +): MdxjsEsm { + return { + type: 'mdxjsEsm', + value: '', + data: { + // @ts-expect-error `parse` return types is incompatible but it should work in runtime + estree: { + ...parse(jsString, acornOpts), + type: 'Program', + sourceType: 'module', + }, + }, + }; +} + +export function ignoreStringPlugins(plugins: any[], logger: AstroIntegrationLogger): PluggableList { + let validPlugins: PluggableList = []; + let hasInvalidPlugin = false; + for (const plugin of plugins) { + if (typeof plugin === 'string') { + logger.warn(`${bold(plugin)} not applied.`); + hasInvalidPlugin = true; + } else if (Array.isArray(plugin) && typeof plugin[0] === 'string') { + logger.warn(`${bold(plugin[0])} not applied.`); + hasInvalidPlugin = true; + } else { + validPlugins.push(plugin); + } + } + if (hasInvalidPlugin) { + logger.warn( + `To inherit Markdown plugins in MDX, please use explicit imports in your config instead of "strings." See Markdown docs: https://docs.astro.build/en/guides/markdown-content/#markdown-plugins`, + ); + } + return validPlugins; +} diff --git a/packages/integrations/mdx/src/vite-plugin-mdx-postprocess.ts b/packages/integrations/mdx/src/vite-plugin-mdx-postprocess.ts new file mode 100644 index 000000000..e00173fbe --- /dev/null +++ b/packages/integrations/mdx/src/vite-plugin-mdx-postprocess.ts @@ -0,0 +1,148 @@ +import type { AstroConfig } from 'astro'; +import { type ExportSpecifier, type ImportSpecifier, parse } from 'es-module-lexer'; +import type { Plugin } from 'vite'; +import { + ASTRO_IMAGE_ELEMENT, + ASTRO_IMAGE_IMPORT, + USES_ASTRO_IMAGE_FLAG, +} from './rehype-images-to-component.js'; +import { type FileInfo, getFileInfo } from './utils.js'; + +const underscoreFragmentImportRegex = /[\s,{]_Fragment[\s,}]/; +const astroTagComponentImportRegex = /[\s,{]__astro_tag_component__[\s,}]/; + +// These transforms must happen *after* JSX runtime transformations +export function vitePluginMdxPostprocess(astroConfig: AstroConfig): Plugin { + return { + name: '@astrojs/mdx-postprocess', + transform(code, id, opts) { + if (!id.endsWith('.mdx')) return; + + const fileInfo = getFileInfo(id, astroConfig); + const [imports, exports] = parse(code); + + // Call a series of functions that transform the code + code = injectUnderscoreFragmentImport(code, imports); + code = injectMetadataExports(code, exports, fileInfo); + code = transformContentExport(code, exports); + code = annotateContentExport(code, id, !!opts?.ssr, imports); + + // The code transformations above are append-only, so the line/column mappings are the same + // and we can omit the sourcemap for performance. + return { code, map: null }; + }, + }; +} + +/** + * Inject `Fragment` identifier import if not already present. + */ +function injectUnderscoreFragmentImport(code: string, imports: readonly ImportSpecifier[]) { + if (!isSpecifierImported(code, imports, underscoreFragmentImportRegex, 'astro/jsx-runtime')) { + code += `\nimport { Fragment as _Fragment } from 'astro/jsx-runtime';`; + } + return code; +} + +/** + * Inject MDX metadata as exports of the module. + */ +function injectMetadataExports( + code: string, + exports: readonly ExportSpecifier[], + fileInfo: FileInfo, +) { + if (!exports.some(({ n }) => n === 'url')) { + code += `\nexport const url = ${JSON.stringify(fileInfo.fileUrl)};`; + } + if (!exports.some(({ n }) => n === 'file')) { + code += `\nexport const file = ${JSON.stringify(fileInfo.fileId)};`; + } + return code; +} + +/** + * Transforms the `MDXContent` default export as `Content`, which wraps `MDXContent` and + * passes additional `components` props. + */ +function transformContentExport(code: string, exports: readonly ExportSpecifier[]) { + if (exports.find(({ n }) => n === 'Content')) return code; + + // If have `export const components`, pass that as props to `Content` as fallback + const hasComponents = exports.find(({ n }) => n === 'components'); + const usesAstroImage = exports.find(({ n }) => n === USES_ASTRO_IMAGE_FLAG); + + // Generate code for the `components` prop passed to `MDXContent` + let componentsCode = `{ Fragment: _Fragment${ + hasComponents ? ', ...components' : '' + }, ...props.components,`; + if (usesAstroImage) { + componentsCode += ` ${JSON.stringify(ASTRO_IMAGE_ELEMENT)}: ${ + hasComponents ? 'components.img ?? ' : '' + } props.components?.img ?? ${ASTRO_IMAGE_IMPORT}`; + } + componentsCode += ' }'; + + // Make `Content` the default export so we can wrap `MDXContent` and pass in `Fragment` + code = code.replace('export default function MDXContent', 'function MDXContent'); + code += ` +export const Content = (props = {}) => MDXContent({ + ...props, + components: ${componentsCode}, +}); +export default Content;`; + return code; +} + +/** + * Add properties to the `Content` export. + */ +function annotateContentExport( + code: string, + id: string, + ssr: boolean, + imports: readonly ImportSpecifier[], +) { + // Mark `Content` as MDX component + code += `\nContent[Symbol.for('mdx-component')] = true`; + // Ensure styles and scripts are injected into a `<head>` when a layout is not applied + code += `\nContent[Symbol.for('astro.needsHeadRendering')] = !Boolean(frontmatter.layout);`; + // Assign the `moduleId` metadata to `Content` + code += `\nContent.moduleId = ${JSON.stringify(id)};`; + + // Tag the `Content` export as "astro:jsx" so it's quicker to identify how to render this component + if (ssr) { + if ( + !isSpecifierImported( + code, + imports, + astroTagComponentImportRegex, + 'astro/runtime/server/index.js', + ) + ) { + code += `\nimport { __astro_tag_component__ } from 'astro/runtime/server/index.js';`; + } + code += `\n__astro_tag_component__(Content, 'astro:jsx');`; + } + + return code; +} + +/** + * Check whether the `specifierRegex` matches for an import of `source` in the `code`. + */ +function isSpecifierImported( + code: string, + imports: readonly ImportSpecifier[], + specifierRegex: RegExp, + source: string, +) { + for (const imp of imports) { + if (imp.n !== source) continue; + + const importStatement = code.slice(imp.ss, imp.se); + if (specifierRegex.test(importStatement)) return true; + } + + return false; +} diff --git a/packages/integrations/mdx/src/vite-plugin-mdx.ts b/packages/integrations/mdx/src/vite-plugin-mdx.ts new file mode 100644 index 000000000..869c65d26 --- /dev/null +++ b/packages/integrations/mdx/src/vite-plugin-mdx.ts @@ -0,0 +1,106 @@ +import type { SSRError } from 'astro'; +import { getAstroMetadata } from 'astro/jsx/rehype.js'; +import { VFile } from 'vfile'; +import type { Plugin } from 'vite'; +import type { MdxOptions } from './index.js'; +import { createMdxProcessor } from './plugins.js'; +import { safeParseFrontmatter } from './utils.js'; + +export interface VitePluginMdxOptions { + mdxOptions: MdxOptions; + srcDir: URL; +} + +// NOTE: Do not destructure `opts` as we're assigning a reference that will be mutated later +export function vitePluginMdx(opts: VitePluginMdxOptions): Plugin { + let processor: ReturnType<typeof createMdxProcessor> | undefined; + let sourcemapEnabled: boolean; + + return { + name: '@mdx-js/rollup', + enforce: 'pre', + buildEnd() { + processor = undefined; + }, + configResolved(resolved) { + sourcemapEnabled = !!resolved.build.sourcemap; + + // HACK: Remove the `astro:jsx` plugin if defined as we handle the JSX transformation ourselves + const jsxPluginIndex = resolved.plugins.findIndex((p) => p.name === 'astro:jsx'); + if (jsxPluginIndex !== -1) { + // @ts-ignore-error ignore readonly annotation + resolved.plugins.splice(jsxPluginIndex, 1); + } + }, + async resolveId(source, importer, options) { + if (importer?.endsWith('.mdx') && source[0] !== '/') { + let resolved = await this.resolve(source, importer, options); + if (!resolved) resolved = await this.resolve('./' + source, importer, options); + return resolved; + } + }, + // Override transform to alter code before MDX compilation + // ex. inject layouts + async transform(code, id) { + if (!id.endsWith('.mdx')) return; + + const { frontmatter, content } = safeParseFrontmatter(code, id); + + const vfile = new VFile({ + value: content, + path: id, + data: { + astro: { + frontmatter, + }, + applyFrontmatterExport: { + srcDir: opts.srcDir, + }, + }, + }); + + // Lazily initialize the MDX processor + if (!processor) { + processor = createMdxProcessor(opts.mdxOptions, { sourcemap: sourcemapEnabled }); + } + + try { + const compiled = await processor.process(vfile); + + return { + code: String(compiled.value), + map: compiled.map, + meta: getMdxMeta(vfile), + }; + } catch (e: any) { + const err: SSRError = e; + + // For some reason MDX puts the error location in the error's name, not very useful for us. + err.name = 'MDXError'; + err.loc = { file: id, line: e.line, column: e.column }; + + // For another some reason, MDX doesn't include a stack trace. Weird + Error.captureStackTrace(err); + + throw err; + } + }, + }; +} + +function getMdxMeta(vfile: VFile): Record<string, any> { + const astroMetadata = getAstroMetadata(vfile); + if (!astroMetadata) { + throw new Error( + 'Internal MDX error: Astro metadata is not set by rehype-analyze-astro-metadata', + ); + } + return { + astro: astroMetadata, + vite: { + // Setting this vite metadata to `ts` causes Vite to resolve .js + // extensions to .ts files. + lang: 'ts', + }, + }; +} |