From 6362414d65b69cd01624e84d08eca654fc8cb101 Mon Sep 17 00:00:00 2001 From: Jarred Sumner Date: Fri, 7 Apr 2023 20:08:01 -0700 Subject: Bun gets a new bundler (#2312) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * alright now just gotta try running it * fix a gajillion compiler errors * even more code * okay i fixed more errors * wip * Update launch.json * Update string_builder.zig * `fast_debug_build_mode` makes debug build 2x faster * Update bundle_v2.zig * more code! * It bundles! * Rename `Bun.Transpiler` to `Bun.Bundler` * `import()` expressions almost work * wip attempt to get import() expr to work * Bundle namespace imports * Attempt to fix the issue with import() unsuccessfully * consider current working directory when resolving relative paths (#2313) * consider current working directory when resolving relative paths fixes #2298 * comment test --------- Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com> * support `expect().toThrow(/pattern/)` (#2314) - fix time-zone-dependent test failure * fix missing `Blob` error messages on Linux (#2315) * fix & clean up tests (#2318) - skip flaky tests when running as `root` - use `expect().toThrow()` - clean up temporary files after tests * feat(tty): add some `tty.WriteStream` methods to `process.{stdout, stderr}` (#2320) * feat(stdio): add some `tty.WriteStream` methods * chore(builtins): add process builtin gen'd code * Fix docker install command * `bun test` on macOS in GitHub Actions (#2322) * Fixes #2323 * throw invalid parameter errors in `crypto.scryptSync` (#2331) * throw invalid parameter errors * remove comptime, add empty buffer function * remove error_name comptime * Add reference documentation for bun:test (#2327) * Reorganize tests (#2332) * Fix html-rewriter.test.js * fix the wrong thing being incremented in hmr example (#2334) * Add more test harness * Improve Benchmarking page, small fixes (#2339) * Improve benchmarking page * WIP * Add typescript instructions to hot * Document preload in Plugins. Fix loader in plugin types. * Fix typo * Fix links * run prettier * Document openInEditor * improve `Buffer` compatibility with Node.js (#2341) * improve `Buffer` compatibility with Node.js * use `memmove()` allow `encoding` to be `undefined` * run `bun test` after macOS builds (#2343) * "binary" is an alias of "latin1" Fixes https://github.com/oven-sh/bun/issues/2110 * More spec compliant `Blob.prototype.type` (#2340) * Make `Blob.prototype. type` more spec compliant * Add a few more checks for isNumber() * Fix `make headers` * Safer JSValue.isString() * More tests for blob.slice * Make `Blob.prototype.type` more spec compliant * Add isASCII check * Fix types * Fix failing type test * Update blob.zig * Update blob.zig * Fix .eql check on empty values --------- Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com> * Fix bug in test runner * Support `import()` expressions * Implement `require()` * clean up bit_set.zig slightly * Move some things around * misc cleanup * Cleanup some things * Fix a lot of stuff * Fix `module.exports.fn = fn;` in ESM entry point * Fix crash due when printing file * Fix issue with class names * Fix issue with `export default identifier` * Update js_parser.zig * optimization: inline single-property object acceses and arrays * Fix undefined memory in renamed symbols list * Handle call target * wip * Inline it * Fix undefined memory issue when reclaiming blocks in ast * Halt linking on any parse errors * alias * Rename `enable_bundling` to `enable_legacy_bundling` * Workaround anonymous struct literal zig bug * Use slower approach (without bitset) because it doesn't break after 8 symbols * Fix incorrectly-renaming statically defined symbols * Handle more edgecases in our bit_set fork * Reduce number of allocations for `define` * Do not rename unbound symbols * Clean up dot defines a little more * Make the generated names prettier * Workaround runtime symbol missing issue * Fail the build on errors * Support export * from * Support `--outfile` * partially fix renaming * fanicer symbol renaming impl * misc, extremely revertible cleanup * Fix up some bugs with symbol renaming * formatting * Update launch.json * Parse `__PURE__` comments * clean up simd code for pure comments * changes to merge * workaround runtime issue * Fix issue with `export * as` not propagating correctly * Make all top-level declarations `var` when bundling * Fix missing prefix * Fix assigning to stack copy * Fix missing runtime symbol * Fix bug with namespace exports * Dramatically reduce allocations * Update launch.json * Add missing flags * Update js_parser.zig * small cleanup * Make the export name better * Fix unnecessary `var foo = foo` * Implement CommonJS -> ESM conversion * Implement module redirects * Port esbuild bundler tests for new bundler (#2380) * started porting esbuild tests * clean up test names and api before moving on * port tests using a program i wrote * replace todo generated comment * fix generated tests not including some files * work on tests * [github web editor] add define, external, inject, minifySyntax, minifyWhitespace options. * get most of the todo comments out of the way, but expectBundled does not handle most of the cases * continue working on esbuild tests * use test.skip for unsupported tests * Fixups for test runner * Hoist imports & exports * Fix test * Hoist classes * bundler test refining, 51/835 * Fix runtime require * bundler test refining, 81/835 * bundler test refining, 93/835 * Make the test work in any timezone * feat(expect): update toBeInstanceOf (#2396) * feat: update instanceof binding * fix: according to PR comments * Rename `expectObjectTypeCount` to `expectMaxObjectTypeCount` * Fix socket tests with connection errors (#2403) * release pending activity with connection error handler * unref poll_ref * remove trailing comma * Organize Dockerfiles for official status * Remove test Dockerfile * Remove old Docker workflow * Feat(test): add toMatch (#2404) * Fix various fetch/response/request tests (#2416) * fix most fetch tests, skip a few * fastGet, toValueGC, and invalid init * bigint unreachable, range error, log process as process * remove extra fetch_headers * remove js_type parameter, check isObject() * throw invalid mime type error, use enum literal * switch back to promise rejection * RangeError pascal case * Fix several bugs (#2418) * utf16 codepoint with replacement character * Fix test failure with `TextEncoder("ascii')` * Add missing type * Fix Response.prototype.bodyUsed and Request.prototype.bodyUsed * Fix bug with scrypt error not clearing * Update server.zig * oopsie * :nail_care: * docs: Use correct url in the 'Issues' link in README header (#2420) * Fix crash when rendering error page and the server or network is slow * [fetch] Make the default body value `null` when unspecified This is better aligned with the fetch spec * Make node-net tests less flaky * [node:net] Fix issue with `listen` callback firing before it's listening * Always clear timers in node test harness * Fix out of bounds access Repro'd in Buffer tests * Update UWS cc @cirospaciari * Make this test more thorough * Hanging abort test * 0 length body is a null stream * Several bug fixes (#2427) * Fix test * Fix segfault when unexpected type is passed in `expect().toThrow` * Fix issues with request constructor * Don't bother cloning headers when its empty * woops * more tests * fix incorrect test * Make the fetch error messages better * Update response.zig * Fix test that failed on macOS * Fix test * Remove extra hash table lookups * Support running dummy registry directly cc @alexlamsl * Update test * Update test * fixup * Workaround crash in test runner * Fixup test * Fixup test * Update os.test.js --------- Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com> * Remove usages of port numbers in tests * Set -O2 and -fno-rtti * Remove -g * Prevent undefined memory access * [bun test] Implement `--rerun-each` flag to run each test N times * Reduce number of module scopes created * add some extra abort checks into streams (#2430) * add some checks to avoid UAF * avoid multiple calls to finalize if endFromJS is called more than once * fix no-op comment * mark as requested_end on abort * remove requested_end from abort * remove unnecessary check (#2432) * Fix bug with scoped aliased dependencies in bun install on macOS * remove `addLog`, remove `--prominent-compile-errors` * Finish the upgrade * Optional chaining flag * Implement same_target_becomes_destructuring optimization * bundler test refining, 109/835 * Reset bindings * Support multiple entry points * Implement `--entry-names` flag * Use a tempdir with a better name * prettier * Log file name * Update js_parser.zig * Mark all bun builtins as external * Make resolve errors actually errors * Update bundler_default.test.ts * Fix `await import(foo)` * WIP react server components * Do more stuff at runtime * :scissors: * Support automatic JSX imports * Use a module cache for now * Update tsconfig.base.json * Fix ThisOutsideFunctionNotRenamed * woopsie * moar cpu * clamp it * fixup * Add a bunch of assertions * Bun uses automatic runtime by default * Parse Import Attributes * Add a note about Valgrind * Update developing.md * Fix up code splitting for React Server Components * Implement client component manifest * Fix crash with --react-server-components and no client components * Backport https://github.com/ziglang/zig/commit/4d31e3c917a05541394c544708f0047cfb53331a * Update launch.json * Fix for latest zig * Workaround bug with ?[]const string Occasionally saw alignment errors in this code Workaround https://github.com/ziglang/zig/issues/15085 related: https://github.com/ziglang/zig/pull/15089 * switch to regular slice * Avoid initializing named_imports and named_exports as undefined * Reduce usages of `undefined` * Add more assertions * --watch wip * Update javascript.zig * Possibly fix the race condition * Faster `do` * bump allocator * Reduce the size of `Symbol` slightly * Alphabetically sort runtime import symbols, for determinism * Prepare for code splitting * handle overlapping stdout * pure * clean up some things * Fix bug with `$$typeof` * Address CommonJS -> ESM hoisting bug * Support `"use server"` in manifest * Implement `"use server"` * Fix importing bun builtins when bundling * Make `commonjs_to_esm` a feature flag, fix some splitting bugs * :scissors: * fixme remove this * Fix crash in longestCommonPath * Chunking! Just need to do import paths now. * Import paths work...now trying to figure out how to make runtime symbols work * add workaround * Replace `bun bun` with `bun build` * Fix crash with dual package hazard * Fix many CommonJS <> ESM interop bugs * Support package.json `"sideEffects"` also skip loading unnecessary package.json data in `bun run` * add a not good --watch implementation * bundler test refining, 140/831 * remove accidentally committed file * do not return status code 1 on successful bundles * bundler test refining, 159/830 * pass exit code to exitOrWatch * clean up help menu -remove two spaces to line up bun build -moved all tags to the end of the text they are colorizing -moved other colors to the start of the text they colorize -removed unneeded tags, keeping only one at the start of the block * importstar is fully ported * wip * you can run code in this branch now * Disable this transform * organize and document bundler tests * Fix double import * Fix sloppy mode function declarations * Disable our CommonJS transform for now * add `assertNotPresent` to make splitting cases easier * Bump! * Update bun.d.ts * use import.meta.require in runtime code * Disable this again * Fix dirname * Fix ESM -> CJS wrapper * :nail_care: --------- Co-authored-by: Jarred Sumner <709451+Jarred-Sumner@users.noreply.github.com> Co-authored-by: Alex Lam S.L Co-authored-by: Derrick Farris Co-authored-by: Ashcon Partovi Co-authored-by: Dylan Conway <35280289+dylan-conway@users.noreply.github.com> Co-authored-by: pfg Co-authored-by: Colin McDonnell Co-authored-by: dave caruso Co-authored-by: zhiyuan <32867472+zhiyuang@users.noreply.github.com> Co-authored-by: Dylan Conway Co-authored-by: Kamil Ogórek Co-authored-by: Ciro Spaciari --- src/bun.js/api/JSBundler.classes.ts | 51 ++ src/bun.js/api/JSBundler.zig | 332 +++++++++ src/bun.js/api/JSTranspiler.zig | 1305 ++++++++++++++++++++++++++++++++++ src/bun.js/api/bun.zig | 18 +- src/bun.js/api/ffi.zig | 1 - src/bun.js/api/server.zig | 8 +- src/bun.js/api/transpiler.classes.ts | 41 -- src/bun.js/api/transpiler.zig | 1305 ---------------------------------- 8 files changed, 1711 insertions(+), 1350 deletions(-) create mode 100644 src/bun.js/api/JSBundler.classes.ts create mode 100644 src/bun.js/api/JSBundler.zig create mode 100644 src/bun.js/api/JSTranspiler.zig delete mode 100644 src/bun.js/api/transpiler.classes.ts delete mode 100644 src/bun.js/api/transpiler.zig (limited to 'src/bun.js/api') diff --git a/src/bun.js/api/JSBundler.classes.ts b/src/bun.js/api/JSBundler.classes.ts new file mode 100644 index 000000000..5eeed30f0 --- /dev/null +++ b/src/bun.js/api/JSBundler.classes.ts @@ -0,0 +1,51 @@ +import { define } from "../scripts/class-definitions"; + +export default [ + define({ + name: "Transpiler", + construct: true, + finalize: true, + hasPendingActivity: false, + configurable: false, + klass: {}, + JSType: "0b11101110", + proto: { + scanImports: { + fn: "scanImports", + length: 2, + }, + scan: { + fn: "scan", + length: 2, + }, + transform: { + fn: "transform", + length: 2, + }, + transformSync: { + fn: "transformSync", + length: 2, + }, + }, + }), + + define({ + name: "Bundler", + construct: true, + finalize: true, + hasPendingActivity: true, + configurable: false, + klass: {}, + JSType: "0b11101110", + proto: { + handle: { + fn: "handleRequest", + length: 2, + }, + write: { + fn: "write", + length: 1, + }, + }, + }), +]; diff --git a/src/bun.js/api/JSBundler.zig b/src/bun.js/api/JSBundler.zig new file mode 100644 index 000000000..e8ce7a0e5 --- /dev/null +++ b/src/bun.js/api/JSBundler.zig @@ -0,0 +1,332 @@ +const std = @import("std"); +const Api = @import("../../api/schema.zig").Api; +const http = @import("../../http.zig"); +const JavaScript = @import("../javascript.zig"); +const QueryStringMap = @import("../../url.zig").QueryStringMap; +const CombinedScanner = @import("../../url.zig").CombinedScanner; +const bun = @import("bun"); +const string = bun.string; +const JSC = bun.JSC; +const js = JSC.C; +const WebCore = @import("../webcore/response.zig"); +const Bundler = bun.bundler; +const options = @import("../../options.zig"); +const VirtualMachine = JavaScript.VirtualMachine; +const ScriptSrcStream = std.io.FixedBufferStream([]u8); +const ZigString = JSC.ZigString; +const Fs = @import("../../fs.zig"); +const Base = @import("../base.zig"); +const getAllocator = Base.getAllocator; +const JSObject = JSC.JSObject; +const JSError = Base.JSError; +const JSValue = bun.JSC.JSValue; +const JSGlobalObject = JSC.JSGlobalObject; +const strings = bun.strings; +const NewClass = Base.NewClass; +const To = Base.To; +const Request = WebCore.Request; + +const FetchEvent = WebCore.FetchEvent; +const MacroMap = @import("../../resolver/package_json.zig").MacroMap; +const TSConfigJSON = @import("../../resolver/tsconfig_json.zig").TSConfigJSON; +const PackageJSON = @import("../../resolver/package_json.zig").PackageJSON; +const logger = bun.logger; +const Loader = options.Loader; +const Platform = options.Platform; +const JSAst = bun.JSAst; +const JSParser = bun.js_parser; +const JSPrinter = bun.js_printer; +const ScanPassResult = JSParser.ScanPassResult; +const Mimalloc = @import("../../mimalloc_arena.zig"); +const Runtime = @import("../../runtime.zig").Runtime; +const JSLexer = bun.js_lexer; +const Expr = JSAst.Expr; + +pub const JSBundler = struct { + heap: Mimalloc.Arena, + allocator: std.mem.Allocator, + configs: Config.List = .{}, + has_pending_activity: std.atomic.Atomic(bool) = std.atomic.Atomic(bool).init(true), + + pub usingnamespace JSC.Codegen.JSBundler; + + const OwnedString = bun.MutableString; + + pub const Config = struct { + target: options.Platform = options.Platform.browser, + entry_points: std.BufSet = std.BufSet.init(bun.default_allocator), + hot: bool = false, + define: std.BufMap = std.BufMap.init(bun.default_allocator), + dir: OwnedString = OwnedString.initEmpty(bun.default_allocator), + serve: Serve = .{}, + jsx: options.JSX.Pragma = .{}, + code_splitting: bool = false, + minify: Minify = .{}, + server_components: ServerComponents = ServerComponents{}, + plugins: PluginDeclaration.List = .{}, + + names: Names = .{}, + label: OwnedString = OwnedString.initEmpty(bun.default_allocator), + + pub const List = bun.StringArrayHashMapUnmanaged(Config); + + /// + /// { name: "", setup: (build) {} } + pub const PluginDeclaration = struct { + name: OwnedString = OwnedString.initEmpty(bun.default_allocator), + setup: JSC.Strong = .{}, + + pub const List = std.ArrayListUnmanaged(PluginDeclaration); + + pub fn deinit(this: *PluginDeclaration) void { + this.name.deinit(); + this.setup.deinit(); + } + }; + + pub fn fromJS(globalThis: *JSC.JSGlobalObject, config: JSC.JSValue, allocator: std.mem.Allocator) !Config { + var this = Config{ + .entry_points = std.BufSet.init(allocator), + .define = std.BufMap.init(allocator), + .dir = OwnedString.initEmpty(allocator), + .label = OwnedString.initEmpty(allocator), + .names = .{ + .owned_entry_point = OwnedString.initEmpty(allocator), + .owned_chunk = OwnedString.initEmpty(allocator), + }, + }; + errdefer this.deinit(allocator); + + if (try config.getOptionalEnum(globalThis, "target", options.Platform)) |target| { + this.target = target; + } + + if (try config.getOptional(globalThis, "hot", bool)) |hot| { + this.hot = hot; + } + + if (try config.getOptional(globalThis, "splitting", bool)) |hot| { + this.code_splitting = hot; + } + + if (try config.getOptional(globalThis, "minifyWhitespace", bool)) |hot| { + this.minify.whitespace = hot; + } + + if (try config.getArray(globalThis, "entrypoints")) |entry_points| { + var iter = entry_points.arrayIterator(globalThis); + while (iter.next()) |entry_point| { + var slice = entry_point.toSliceOrNull(globalThis, allocator) orelse { + globalThis.throwInvalidArguments("Expected entrypoints to be an array of strings", .{}); + return error.JSException; + }; + defer slice.deinit(); + try this.entry_points.insert(slice.slice()); + } + } else { + globalThis.throwInvalidArguments("Expected entrypoints to be an array of strings", .{}); + return error.JSException; + } + + if (try config.getOptional(globalThis, "name", ZigString.Slice)) |slice| { + defer slice.deinit(); + this.label.appendSliceExact(slice.slice()) catch unreachable; + } + + if (try config.getOptional(globalThis, "dir", ZigString.Slice)) |slice| { + defer slice.deinit(); + this.dir.appendSliceExact(slice.slice()) catch unreachable; + } else { + this.dir.appendSliceExact(globalThis.bunVM().bundler.fs.top_level_dir) catch unreachable; + } + + if (try config.getOptional(globalThis, "entryNames", ZigString.Slice)) |slice| { + defer slice.deinit(); + this.names.owned_entry_point.appendSliceExact(slice.slice()) catch unreachable; + this.names.entry_point.data = this.names.owned_entry_point.list.items; + } + + if (try config.getOptional(globalThis, "chunkNames", ZigString.Slice)) |slice| { + defer slice.deinit(); + this.names.owned_chunk.appendSliceExact(slice.slice()) catch unreachable; + this.names.chunk.data = this.names.owned_chunk.list.items; + } + + if (try config.getArray(globalThis, "plugins")) |array| { + var iter = array.arrayIterator(globalThis); + while (iter.next()) |plugin| { + var decl = PluginDeclaration{ + .name = OwnedString.initEmpty(allocator), + .setup = .{}, + }; + errdefer decl.deinit(); + + if (plugin.getObject(globalThis, "SECRET_SERVER_COMPONENTS_INTERNALS")) |internals| { + if (try internals.get(globalThis, "router")) |router_value| { + if (router_value.as(JSC.API.FileSystemRouter) != null) { + this.server_components.router.set(globalThis, router_value); + } else { + globalThis.throwInvalidArguments("Expected router to be a Bun.FileSystemRouter", .{}); + return error.JSError; + } + } + + const directive_object = (try internals.getObject(globalThis, "directive")) orelse { + globalThis.throwInvalidArguments("Expected directive to be an object", .{}); + return error.JSError; + }; + + if (try directive_object.getArray(globalThis, "client")) |client_names_array| { + var array_iter = client_names_array.arrayIterator(globalThis); + while (array_iter.next()) |client_name| { + var slice = client_name.toSliceOrNull(globalThis, allocator) orelse { + globalThis.throwInvalidArguments("Expected directive.client to be an array of strings", .{}); + return error.JSException; + }; + defer slice.deinit(); + try this.server_components.client.append(allocator, OwnedString.initCopy(allocator, slice.slice()) catch unreachable); + } + } else { + globalThis.throwInvalidArguments("Expected directive.client to be an array of strings", .{}); + return error.JSException; + } + + if (try directive_object.getArray(globalThis, "server")) |server_names_array| { + var array_iter = server_names_array.arrayIterator(globalThis); + while (array_iter.next()) |server_name| { + var slice = server_name.toSliceOrNull(globalThis, allocator) orelse { + globalThis.throwInvalidArguments("Expected directive.server to be an array of strings", .{}); + return error.JSException; + }; + defer slice.deinit(); + try this.server_components.server.append(allocator, OwnedString.initCopy(allocator, slice.slice()) catch unreachable); + } + } else { + globalThis.throwInvalidArguments("Expected directive.server to be an array of strings", .{}); + return error.JSException; + } + } + + if (try plugin.getOptional(globalThis, "name", ZigString.Slice)) |slice| { + defer slice.deinit(); + decl.name.appendSliceExact(slice.slice()) catch unreachable; + } + + if (try plugin.getFunction(globalThis, "setup", JSC.JSValue)) |setup| { + decl.setup.set(globalThis, setup); + } else { + globalThis.throwInvalidArguments("Expected plugin to have a setup() function", .{}); + return error.JSError; + } + + try this.plugins.append(allocator, decl); + } + } + + return config; + } + + pub const Names = struct { + owned_entry_point: OwnedString = OwnedString.initEmpty(bun.default_allocator), + entry_point: options.PathTemplate = options.PathTemplate.file, + owned_chunk: OwnedString = OwnedString.initEmpty(bun.default_allocator), + chunk: options.PathTemplate = options.PathTemplate.chunk, + + pub fn deinit(self: *Names) void { + self.owned_entry_point.deinit(); + self.owned_chunk.deinit(); + } + }; + + pub const ServerComponents = struct { + router: JSC.Strong = .{}, + client: std.ArrayListUnmanaged(OwnedString) = .{}, + server: std.ArrayListUnmanaged(OwnedString) = .{}, + + pub fn deinit(self: *ServerComponents, allocator: std.mem.Allocator) void { + self.router.deinit(); + self.client.clearAndFree(allocator); + self.server.clearAndFree(allocator); + } + }; + + pub const Minify = struct { + whitespace: bool = false, + }; + + pub const Serve = struct { + handler_path: OwnedString = OwnedString.initEmpty(bun.default_allocator), + prefix: OwnedString = OwnedString.initEmpty(bun.default_allocator), + + pub fn deinit(self: *Serve, allocator: std.mem.Allocator) void { + _ = allocator; + self.handler_path.deinit(); + self.prefix.deinit(); + } + }; + + pub fn deinit(self: *Config, allocator: std.mem.Allocator) void { + self.entry_points.deinit(); + self.define.deinit(); + self.dir.deinit(); + self.serve.deinit(allocator); + self.plugins.deinit(); + self.server_components.deinit(allocator); + self.names.deinit(); + self.label.deinit(); + } + }; + + pub fn constructor( + globalThis: *JSC.JSGlobalObject, + callframe: *JSC.CallFrame, + ) callconv(.C) ?*JSBundler { + var temp = std.heap.ArenaAllocator.init(getAllocator(globalThis)); + const arguments = callframe.arguments(3); + var args = JSC.Node.ArgumentsSlice.init( + globalThis.bunVM(), + arguments.ptr[0..arguments.len], + ); + _ = args; + + defer temp.deinit(); + + return null; + } + + pub fn handleRequest( + this: *JSBundler, + globalThis: *JSC.JSGlobalObject, + callframe: *JSC.CallFrame, + ) callconv(.C) JSValue { + _ = callframe; + _ = globalThis; + _ = this; + + return .zero; + } + + pub fn write( + this: *JSBundler, + globalThis: *JSC.JSGlobalObject, + callframe: *JSC.CallFrame, + ) callconv(.C) JSValue { + _ = callframe; + _ = globalThis; + _ = this; + + return .zero; + } + + pub fn hasPendingActivity(this: *JSBundler) callconv(.C) bool { + @fence(.Acquire); + return this.has_pending_activity.load(.Acquire); + } + + pub fn finalize( + this: *JSBundler, + ) callconv(.C) void { + this.heap.deinit(); + JSC.VirtualMachine.get().allocator.destroy(this); + } +}; diff --git a/src/bun.js/api/JSTranspiler.zig b/src/bun.js/api/JSTranspiler.zig new file mode 100644 index 000000000..465ab0ad4 --- /dev/null +++ b/src/bun.js/api/JSTranspiler.zig @@ -0,0 +1,1305 @@ +const std = @import("std"); +const Api = @import("../../api/schema.zig").Api; +const http = @import("../../http.zig"); +const JavaScript = @import("../javascript.zig"); +const QueryStringMap = @import("../../url.zig").QueryStringMap; +const CombinedScanner = @import("../../url.zig").CombinedScanner; +const bun = @import("bun"); +const string = bun.string; +const JSC = bun.JSC; +const js = JSC.C; +const WebCore = @import("../webcore/response.zig"); +const Bundler = bun.bundler; +const options = @import("../../options.zig"); +const VirtualMachine = JavaScript.VirtualMachine; +const ScriptSrcStream = std.io.FixedBufferStream([]u8); +const ZigString = JSC.ZigString; +const Fs = @import("../../fs.zig"); +const Base = @import("../base.zig"); +const getAllocator = Base.getAllocator; +const JSObject = JSC.JSObject; +const JSError = Base.JSError; +const JSValue = bun.JSC.JSValue; +const JSGlobalObject = JSC.JSGlobalObject; +const strings = bun.strings; +const NewClass = Base.NewClass; +const To = Base.To; +const Request = WebCore.Request; + +const FetchEvent = WebCore.FetchEvent; +const MacroMap = @import("../../resolver/package_json.zig").MacroMap; +const TSConfigJSON = @import("../../resolver/tsconfig_json.zig").TSConfigJSON; +const PackageJSON = @import("../../resolver/package_json.zig").PackageJSON; +const logger = bun.logger; +const Loader = options.Loader; +const Platform = options.Platform; +const JSAst = bun.JSAst; +const Transpiler = @This(); +const JSParser = bun.js_parser; +const JSPrinter = bun.js_printer; +const ScanPassResult = JSParser.ScanPassResult; +const Mimalloc = @import("../../mimalloc_arena.zig"); +const Runtime = @import("../../runtime.zig").Runtime; +const JSLexer = bun.js_lexer; +const Expr = JSAst.Expr; + +pub usingnamespace JSC.Codegen.JSTranspiler; + +bundler: Bundler.Bundler, +arena: std.heap.ArenaAllocator, +transpiler_options: TranspilerOptions, +scan_pass_result: ScanPassResult, +buffer_writer: ?JSPrinter.BufferWriter = null, + +const default_transform_options: Api.TransformOptions = brk: { + var opts = std.mem.zeroes(Api.TransformOptions); + opts.disable_hmr = true; + opts.platform = Api.Platform.browser; + opts.serve = false; + break :brk opts; +}; + +const TranspilerOptions = struct { + transform: Api.TransformOptions = default_transform_options, + default_loader: options.Loader = options.Loader.jsx, + macro_map: MacroMap = MacroMap{}, + tsconfig: ?*TSConfigJSON = null, + tsconfig_buf: []const u8 = "", + macros_buf: []const u8 = "", + log: logger.Log, + runtime: Runtime.Features = Runtime.Features{ .top_level_await = true }, + tree_shaking: bool = false, + trim_unused_imports: ?bool = null, + inlining: bool = false, + + minify_whitespace: bool = false, + minify_identifiers: bool = false, +}; + +// Mimalloc gets unstable if we try to move this to a different thread +// threadlocal var transform_buffer: bun.MutableString = undefined; +// threadlocal var transform_buffer_loaded: bool = false; + +// This is going to be hard to not leak +pub const TransformTask = struct { + input_code: ZigString = ZigString.init(""), + protected_input_value: JSC.JSValue = @intToEnum(JSC.JSValue, 0), + output_code: ZigString = ZigString.init(""), + bundler: Bundler.Bundler = undefined, + log: logger.Log, + err: ?anyerror = null, + macro_map: MacroMap = MacroMap{}, + tsconfig: ?*TSConfigJSON = null, + loader: Loader, + global: *JSGlobalObject, + replace_exports: Runtime.Features.ReplaceableExport.Map = .{}, + + pub const AsyncTransformTask = JSC.ConcurrentPromiseTask(TransformTask); + pub const AsyncTransformEventLoopTask = AsyncTransformTask.EventLoopTask; + + pub fn create(transpiler: *Transpiler, protected_input_value: JSC.JSValue, globalThis: *JSGlobalObject, input_code: ZigString, loader: Loader) !*AsyncTransformTask { + var transform_task = try bun.default_allocator.create(TransformTask); + transform_task.* = .{ + .input_code = input_code, + .protected_input_value = protected_input_value, + .bundler = undefined, + .global = globalThis, + .macro_map = transpiler.transpiler_options.macro_map, + .tsconfig = transpiler.transpiler_options.tsconfig, + .log = logger.Log.init(bun.default_allocator), + .loader = loader, + .replace_exports = transpiler.transpiler_options.runtime.replace_exports, + }; + transform_task.bundler = transpiler.bundler; + transform_task.bundler.linker.resolver = &transform_task.bundler.resolver; + + transform_task.bundler.setLog(&transform_task.log); + transform_task.bundler.setAllocator(bun.default_allocator); + return try AsyncTransformTask.createOnJSThread(bun.default_allocator, globalThis, transform_task); + } + + pub fn run(this: *TransformTask) void { + const name = this.loader.stdinName(); + const source = logger.Source.initPathString(name, this.input_code.slice()); + + JSAst.Stmt.Data.Store.create(bun.default_allocator); + JSAst.Expr.Data.Store.create(bun.default_allocator); + + var arena = Mimalloc.Arena.init() catch unreachable; + + const allocator = arena.allocator(); + + defer { + JSAst.Stmt.Data.Store.reset(); + JSAst.Expr.Data.Store.reset(); + arena.deinit(); + } + + this.bundler.setAllocator(allocator); + const jsx = if (this.tsconfig != null) + this.tsconfig.?.mergeJSX(this.bundler.options.jsx) + else + this.bundler.options.jsx; + + const parse_options = Bundler.Bundler.ParseOptions{ + .allocator = allocator, + .macro_remappings = this.macro_map, + .dirname_fd = 0, + .file_descriptor = null, + .loader = this.loader, + .jsx = jsx, + .path = source.path, + .virtual_source = &source, + .replace_exports = this.replace_exports, + // .allocator = this. + }; + + const parse_result = this.bundler.parse(parse_options, null) orelse { + this.err = error.ParseError; + return; + }; + + if (parse_result.empty) { + this.output_code = ZigString.init(""); + return; + } + + var global_allocator = arena.backingAllocator(); + var buffer_writer = JSPrinter.BufferWriter.init(global_allocator) catch |err| { + this.err = err; + return; + }; + buffer_writer.buffer.list.ensureTotalCapacity(global_allocator, 512) catch unreachable; + buffer_writer.reset(); + + // defer { + // transform_buffer = buffer_writer.buffer; + // } + + var printer = JSPrinter.BufferPrinter.init(buffer_writer); + const printed = this.bundler.print(parse_result, @TypeOf(&printer), &printer, .esm_ascii) catch |err| { + this.err = err; + return; + }; + + if (printed > 0) { + buffer_writer = printer.ctx; + buffer_writer.buffer.list.items = buffer_writer.written; + + var output = JSC.ZigString.init(buffer_writer.written); + output.mark(); + this.output_code = output; + } else { + this.output_code = ZigString.init(""); + } + } + + pub fn then(this: *TransformTask, promise: *JSC.JSPromise) void { + if (this.log.hasAny() or this.err != null) { + const error_value: JSValue = brk: { + if (this.err) |err| { + if (!this.log.hasAny()) { + break :brk JSC.JSValue.fromRef(JSC.BuildError.create( + this.global, + bun.default_allocator, + logger.Msg{ + .data = logger.Data{ .text = bun.asByteSlice(@errorName(err)) }, + }, + )); + } + } + + break :brk this.log.toJS(this.global, bun.default_allocator, "Transform failed"); + }; + + promise.reject(this.global, error_value); + return; + } + + finish(this.output_code, this.global, promise); + + if (@enumToInt(this.protected_input_value) != 0) { + this.protected_input_value = @intToEnum(JSC.JSValue, 0); + } + this.deinit(); + } + + noinline fn finish(code: ZigString, global: *JSGlobalObject, promise: *JSC.JSPromise) void { + promise.resolve(global, code.toValueGC(global)); + } + + pub fn deinit(this: *TransformTask) void { + var should_cleanup = false; + defer if (should_cleanup) bun.Global.mimalloc_cleanup(false); + + this.log.deinit(); + if (this.input_code.isGloballyAllocated()) { + this.input_code.deinitGlobal(); + } + + if (this.output_code.isGloballyAllocated()) { + should_cleanup = this.output_code.len > 512_000; + this.output_code.deinitGlobal(); + } + + bun.default_allocator.destroy(this); + } +}; + +fn exportReplacementValue(value: JSValue, globalThis: *JSGlobalObject) ?JSAst.Expr { + if (value.isBoolean()) { + return Expr{ + .data = .{ + .e_boolean = .{ + .value = value.toBoolean(), + }, + }, + .loc = logger.Loc.Empty, + }; + } + + if (value.isNumber()) { + return Expr{ + .data = .{ + .e_number = .{ .value = value.asNumber() }, + }, + .loc = logger.Loc.Empty, + }; + } + + if (value.isNull()) { + return Expr{ + .data = .{ + .e_null = .{}, + }, + .loc = logger.Loc.Empty, + }; + } + + if (value.isUndefined()) { + return Expr{ + .data = .{ + .e_undefined = .{}, + }, + .loc = logger.Loc.Empty, + }; + } + + if (value.isString()) { + var str = JSAst.E.String{ + .data = std.fmt.allocPrint(bun.default_allocator, "{}", .{value.getZigString(globalThis)}) catch unreachable, + }; + var out = bun.default_allocator.create(JSAst.E.String) catch unreachable; + out.* = str; + return Expr{ + .data = .{ + .e_string = out, + }, + .loc = logger.Loc.Empty, + }; + } + + return null; +} + +fn transformOptionsFromJSC(globalObject: JSC.C.JSContextRef, temp_allocator: std.mem.Allocator, args: *JSC.Node.ArgumentsSlice, exception: JSC.C.ExceptionRef) !TranspilerOptions { + var globalThis = globalObject; + const object = args.next() orelse return TranspilerOptions{ .log = logger.Log.init(temp_allocator) }; + if (object.isUndefinedOrNull()) return TranspilerOptions{ .log = logger.Log.init(temp_allocator) }; + + args.eat(); + var allocator = args.arena.allocator(); + + var transpiler = TranspilerOptions{ + .default_loader = .jsx, + .transform = default_transform_options, + .log = logger.Log.init(allocator), + }; + transpiler.log.level = .warn; + + if (!object.isObject()) { + JSC.throwInvalidArguments("Expected an object", .{}, globalObject, exception); + return transpiler; + } + + if (object.getIfPropertyExists(globalObject, "define")) |define| { + define: { + if (define.isUndefinedOrNull()) { + break :define; + } + + if (!define.isObject()) { + JSC.throwInvalidArguments("define must be an object", .{}, globalObject, exception); + return transpiler; + } + + var define_iter = JSC.JSPropertyIterator(.{ + .skip_empty_name = true, + + .include_value = true, + }).init(globalThis, define.asObjectRef()); + defer define_iter.deinit(); + + // cannot be a temporary because it may be loaded on different threads. + var map_entries = allocator.alloc([]u8, define_iter.len * 2) catch unreachable; + var names = map_entries[0..define_iter.len]; + + var values = map_entries[define_iter.len..]; + + while (define_iter.next()) |prop| { + const property_value = define_iter.value; + const value_type = property_value.jsType(); + + if (!value_type.isStringLike()) { + JSC.throwInvalidArguments("define \"{s}\" must be a JSON string", .{prop}, globalObject, exception); + return transpiler; + } + + names[define_iter.i] = prop.toOwnedSlice(allocator) catch unreachable; + var val = JSC.ZigString.init(""); + property_value.toZigString(&val, globalThis); + if (val.len == 0) { + val = JSC.ZigString.init("\"\""); + } + values[define_iter.i] = std.fmt.allocPrint(allocator, "{}", .{val}) catch unreachable; + } + + transpiler.transform.define = Api.StringMap{ + .keys = names, + .values = values, + }; + } + } + + if (object.get(globalThis, "external")) |external| { + external: { + if (external.isUndefinedOrNull()) break :external; + + const toplevel_type = external.jsType(); + if (toplevel_type.isStringLike()) { + var zig_str = JSC.ZigString.init(""); + external.toZigString(&zig_str, globalThis); + if (zig_str.len == 0) break :external; + var single_external = allocator.alloc(string, 1) catch unreachable; + single_external[0] = std.fmt.allocPrint(allocator, "{}", .{external}) catch unreachable; + transpiler.transform.external = single_external; + } else if (toplevel_type.isArray()) { + const count = external.getLengthOfArray(globalThis); + if (count == 0) break :external; + + var externals = allocator.alloc(string, count) catch unreachable; + var iter = external.arrayIterator(globalThis); + var i: usize = 0; + while (iter.next()) |entry| { + if (!entry.jsType().isStringLike()) { + JSC.throwInvalidArguments("external must be a string or string[]", .{}, globalObject, exception); + return transpiler; + } + + var zig_str = JSC.ZigString.init(""); + entry.toZigString(&zig_str, globalThis); + if (zig_str.len == 0) continue; + externals[i] = std.fmt.allocPrint(allocator, "{}", .{external}) catch unreachable; + i += 1; + } + + transpiler.transform.external = externals[0..i]; + } else { + JSC.throwInvalidArguments("external must be a string or string[]", .{}, globalObject, exception); + return transpiler; + } + } + } + + if (object.get(globalThis, "loader")) |loader| { + if (Loader.fromJS(globalThis, loader, exception)) |resolved| { + if (!resolved.isJavaScriptLike()) { + JSC.throwInvalidArguments("only JavaScript-like loaders supported for now", .{}, globalObject, exception); + return transpiler; + } + + transpiler.default_loader = resolved; + } + + if (exception.* != null) { + return transpiler; + } + } + + if (object.get(globalThis, "platform")) |platform| { + if (Platform.fromJS(globalThis, platform, exception)) |resolved| { + transpiler.transform.platform = resolved.toAPI(); + } + + if (exception.* != null) { + return transpiler; + } + } + + if (object.get(globalThis, "tsconfig")) |tsconfig| { + tsconfig: { + if (tsconfig.isUndefinedOrNull()) break :tsconfig; + const kind = tsconfig.jsType(); + var out = JSC.ZigString.init(""); + + if (kind.isArray()) { + JSC.throwInvalidArguments("tsconfig must be a string or object", .{}, globalObject, exception); + return transpiler; + } + + if (!kind.isStringLike()) { + tsconfig.jsonStringify(globalThis, 0, &out); + } else { + tsconfig.toZigString(&out, globalThis); + } + + if (out.len == 0) break :tsconfig; + transpiler.tsconfig_buf = std.fmt.allocPrint(allocator, "{}", .{out}) catch unreachable; + + // TODO: JSC -> Ast conversion + if (TSConfigJSON.parse( + allocator, + &transpiler.log, + logger.Source.initPathString("tsconfig.json", transpiler.tsconfig_buf), + &VirtualMachine.get().bundler.resolver.caches.json, + true, + ) catch null) |parsed_tsconfig| { + transpiler.tsconfig = parsed_tsconfig; + } + } + } + + transpiler.runtime.allow_runtime = false; + transpiler.runtime.dynamic_require = switch (transpiler.transform.platform orelse .browser) { + .bun, .bun_macro => true, + else => false, + }; + + if (object.getIfPropertyExists(globalThis, "macro")) |macros| { + macros: { + if (macros.isUndefinedOrNull()) break :macros; + const kind = macros.jsType(); + const is_object = kind.isObject(); + if (!(kind.isStringLike() or is_object)) { + JSC.throwInvalidArguments("macro must be an object", .{}, globalObject, exception); + return transpiler; + } + + var out: ZigString = ZigString.init(""); + // TODO: write a converter between JSC types and Bun AST types + if (is_object) { + macros.jsonStringify(globalThis, 0, &out); + } else { + macros.toZigString(&out, globalThis); + } + + if (out.len == 0) break :macros; + transpiler.macros_buf = std.fmt.allocPrint(allocator, "{}", .{out}) catch unreachable; + const source = logger.Source.initPathString("macros.json", transpiler.macros_buf); + const json = (VirtualMachine.get().bundler.resolver.caches.json.parseJSON( + &transpiler.log, + source, + allocator, + ) catch null) orelse break :macros; + transpiler.macro_map = PackageJSON.parseMacrosJSON(allocator, json, &transpiler.log, &source); + } + } + + if (object.get(globalThis, "autoImportJSX")) |flag| { + transpiler.runtime.auto_import_jsx = flag.toBoolean(); + } + + if (object.get(globalThis, "allowBunRuntime")) |flag| { + transpiler.runtime.allow_runtime = flag.toBoolean(); + } + + if (object.get(globalThis, "jsxOptimizationInline")) |flag| { + transpiler.runtime.jsx_optimization_inline = flag.toBoolean(); + } + + if (object.get(globalThis, "jsxOptimizationHoist")) |flag| { + transpiler.runtime.jsx_optimization_hoist = flag.toBoolean(); + + if (!transpiler.runtime.jsx_optimization_inline and transpiler.runtime.jsx_optimization_hoist) { + JSC.throwInvalidArguments("jsxOptimizationHoist requires jsxOptimizationInline", .{}, globalObject, exception); + return transpiler; + } + } + + if (object.get(globalThis, "inline")) |flag| { + transpiler.runtime.inlining = flag.toBoolean(); + } + + if (object.get(globalThis, "minifyWhitespace")) |flag| { + transpiler.minify_whitespace = flag.toBoolean(); + } + + if (object.get(globalThis, "sourcemap")) |flag| { + if (flag.isBoolean() or flag.isUndefinedOrNull()) { + if (flag.toBoolean()) { + transpiler.transform.source_map = Api.SourceMapMode.external; + } else { + transpiler.transform.source_map = Api.SourceMapMode.inline_into_file; + } + } else { + var sourcemap = flag.toSlice(globalThis, allocator); + if (options.SourceMapOption.map.get(sourcemap.slice())) |source| { + transpiler.transform.source_map = source.toAPI(); + } else { + JSC.throwInvalidArguments("sourcemap must be one of \"inline\", \"external\", or \"none\"", .{}, globalObject, exception); + return transpiler; + } + } + } + + var tree_shaking: ?bool = null; + if (object.get(globalThis, "treeShaking")) |treeShaking| { + tree_shaking = treeShaking.toBoolean(); + } + + var trim_unused_imports: ?bool = null; + if (object.get(globalThis, "trimUnusedImports")) |trimUnusedImports| { + trim_unused_imports = trimUnusedImports.toBoolean(); + } + + if (object.getTruthy(globalThis, "exports")) |exports| { + if (!exports.isObject()) { + JSC.throwInvalidArguments("exports must be an object", .{}, globalObject, exception); + return transpiler; + } + + var replacements = Runtime.Features.ReplaceableExport.Map{}; + errdefer replacements.clearAndFree(bun.default_allocator); + + if (exports.getTruthy(globalThis, "eliminate")) |eliminate| { + if (!eliminate.jsType().isArray()) { + JSC.throwInvalidArguments("exports.eliminate must be an array", .{}, globalObject, exception); + return transpiler; + } + + var total_name_buf_len: u32 = 0; + var string_count: u32 = 0; + var iter = JSC.JSArrayIterator.init(eliminate, globalThis); + { + var length_iter = iter; + while (length_iter.next()) |value| { + if (value.isString()) { + const length = @truncate(u32, value.getLengthOfArray(globalThis)); + string_count += @as(u32, @boolToInt(length > 0)); + total_name_buf_len += length; + } + } + } + + if (total_name_buf_len > 0) { + var buf = try std.ArrayListUnmanaged(u8).initCapacity(bun.default_allocator, total_name_buf_len); + try replacements.ensureUnusedCapacity(bun.default_allocator, string_count); + { + var length_iter = iter; + while (length_iter.next()) |value| { + if (!value.isString()) continue; + var str = value.getZigString(globalThis); + if (str.len == 0) continue; + const name = std.fmt.bufPrint(buf.items.ptr[buf.items.len..buf.capacity], "{}", .{str}) catch { + JSC.throwInvalidArguments("Error reading exports.eliminate. TODO: utf-16", .{}, globalObject, exception); + return transpiler; + }; + buf.items.len += name.len; + if (name.len > 0) { + replacements.putAssumeCapacity(name, .{ .delete = {} }); + } + } + } + } + } + + if (exports.getTruthy(globalThis, "replace")) |replace| { + if (!replace.isObject()) { + JSC.throwInvalidArguments("replace must be an object", .{}, globalObject, exception); + return transpiler; + } + + var iter = JSC.JSPropertyIterator(.{ + .skip_empty_name = true, + .include_value = true, + }).init(globalThis, replace.asObjectRef()); + + if (iter.len > 0) { + errdefer iter.deinit(); + try replacements.ensureUnusedCapacity(bun.default_allocator, iter.len); + + // We cannot set the exception before `try` because it could be + // a double free with the `errdefer`. + defer if (exception.* != null) { + iter.deinit(); + for (replacements.keys()) |key| { + bun.default_allocator.free(bun.constStrToU8(key)); + } + replacements.clearAndFree(bun.default_allocator); + }; + + while (iter.next()) |key_| { + const value = iter.value; + if (value.isEmpty()) continue; + + var key = try key_.toOwnedSlice(bun.default_allocator); + + if (!JSLexer.isIdentifier(key)) { + JSC.throwInvalidArguments("\"{s}\" is not a valid ECMAScript identifier", .{key}, globalObject, exception); + bun.default_allocator.free(key); + return transpiler; + } + + var entry = replacements.getOrPutAssumeCapacity(key); + + if (exportReplacementValue(value, globalThis)) |expr| { + entry.value_ptr.* = .{ .replace = expr }; + continue; + } + + if (value.isObject() and value.getLengthOfArray(globalObject) == 2) { + const replacementValue = JSC.JSObject.getIndex(value, globalThis, 1); + if (exportReplacementValue(replacementValue, globalThis)) |to_replace| { + const replacementKey = JSC.JSObject.getIndex(value, globalThis, 0); + var slice = (try replacementKey.toSlice(globalThis, bun.default_allocator).cloneIfNeeded(bun.default_allocator)); + var replacement_name = slice.slice(); + + if (!JSLexer.isIdentifier(replacement_name)) { + JSC.throwInvalidArguments("\"{s}\" is not a valid ECMAScript identifier", .{replacement_name}, globalObject, exception); + slice.deinit(); + return transpiler; + } + + entry.value_ptr.* = .{ + .inject = .{ + .name = replacement_name, + .value = to_replace, + }, + }; + continue; + } + } + + JSC.throwInvalidArguments("exports.replace values can only be string, null, undefined, number or boolean", .{}, globalObject, exception); + return transpiler; + } + } + } + + tree_shaking = tree_shaking orelse (replacements.count() > 0); + transpiler.runtime.replace_exports = replacements; + } + + transpiler.tree_shaking = tree_shaking orelse false; + transpiler.trim_unused_imports = trim_unused_imports orelse transpiler.tree_shaking; + + return transpiler; +} + +pub fn constructor( + globalThis: *JSC.JSGlobalObject, + callframe: *JSC.CallFrame, +) callconv(.C) ?*Transpiler { + var temp = std.heap.ArenaAllocator.init(getAllocator(globalThis)); + const arguments = callframe.arguments(3); + var args = JSC.Node.ArgumentsSlice.init( + globalThis.bunVM(), + arguments.ptr[0..arguments.len], + ); + + defer temp.deinit(); + var exception_ref = [_]JSC.C.JSValueRef{null}; + var exception = &exception_ref[0]; + const transpiler_options: TranspilerOptions = if (arguments.len > 0) + transformOptionsFromJSC(globalThis, temp.allocator(), &args, exception) catch { + JSC.throwInvalidArguments("Failed to create transpiler", .{}, globalThis, exception); + return null; + } + else + TranspilerOptions{ .log = logger.Log.init(getAllocator(globalThis)) }; + + if (exception.* != null) { + globalThis.throwValue(JSC.JSValue.c(exception.*)); + return null; + } + + const allocator = getAllocator(globalThis); + + if ((transpiler_options.log.warnings + transpiler_options.log.errors) > 0) { + globalThis.throwValue( + transpiler_options.log.toJS(globalThis.ptr(), allocator, "Failed to create transpiler"), + ); + + return null; + } + + var log = allocator.create(logger.Log) catch unreachable; + log.* = transpiler_options.log; + var bundler = Bundler.Bundler.init( + allocator, + log, + transpiler_options.transform, + null, + JavaScript.VirtualMachine.get().bundler.env, + ) catch |err| { + if ((log.warnings + log.errors) > 0) { + globalThis.throwValue( + log.toJS(globalThis.ptr(), allocator, "Failed to create transpiler"), + ); + + return null; + } + + globalThis.throwError(err, "Error creating transpiler"); + return null; + }; + + bundler.configureLinkerWithAutoJSX(false); + bundler.options.env.behavior = .disable; + bundler.configureDefines() catch |err| { + if ((log.warnings + log.errors) > 0) { + globalThis.throwValue( + log.toJS(globalThis.ptr(), allocator, "Failed to load define"), + ); + + return null; + } + + globalThis.throwError(err, "Failed to load define"); + return null; + }; + + if (transpiler_options.macro_map.count() > 0) { + bundler.options.macro_remap = transpiler_options.macro_map; + } + + bundler.options.minify_whitespace = transpiler_options.minify_whitespace; + bundler.options.tree_shaking = transpiler_options.tree_shaking; + bundler.options.trim_unused_imports = transpiler_options.trim_unused_imports; + bundler.options.allow_runtime = transpiler_options.runtime.allow_runtime; + bundler.options.auto_import_jsx = transpiler_options.runtime.auto_import_jsx; + bundler.options.inlining = transpiler_options.runtime.inlining; + bundler.options.hot_module_reloading = transpiler_options.runtime.hot_module_reloading; + bundler.options.jsx.supports_fast_refresh = bundler.options.hot_module_reloading and + bundler.options.allow_runtime and transpiler_options.runtime.react_fast_refresh; + + var transpiler = allocator.create(Transpiler) catch unreachable; + transpiler.* = Transpiler{ + .transpiler_options = transpiler_options, + .bundler = bundler, + .arena = args.arena, + .scan_pass_result = ScanPassResult.init(allocator), + }; + + return transpiler; +} + +pub fn finalize( + this: *Transpiler, +) callconv(.C) void { + this.bundler.log.deinit(); + this.scan_pass_result.named_imports.deinit(); + this.scan_pass_result.import_records.deinit(); + this.scan_pass_result.used_symbols.deinit(); + if (this.buffer_writer != null) { + this.buffer_writer.?.buffer.deinit(); + } + + // bun.default_allocator.free(this.transpiler_options.tsconfig_buf); + // bun.default_allocator.free(this.transpiler_options.macros_buf); + this.arena.deinit(); + JSC.VirtualMachine.get().allocator.destroy(this); +} + +fn getParseResult(this: *Transpiler, allocator: std.mem.Allocator, code: []const u8, loader: ?Loader, macro_js_ctx: Bundler.MacroJSValueType) ?Bundler.ParseResult { + const name = this.transpiler_options.default_loader.stdinName(); + const source = logger.Source.initPathString(name, code); + + const jsx = if (this.transpiler_options.tsconfig != null) + this.transpiler_options.tsconfig.?.mergeJSX(this.bundler.options.jsx) + else + this.bundler.options.jsx; + + const parse_options = Bundler.Bundler.ParseOptions{ + .allocator = allocator, + .macro_remappings = this.transpiler_options.macro_map, + .dirname_fd = 0, + .file_descriptor = null, + .loader = loader orelse this.transpiler_options.default_loader, + .jsx = jsx, + .path = source.path, + .virtual_source = &source, + .replace_exports = this.transpiler_options.runtime.replace_exports, + .macro_js_ctx = macro_js_ctx, + // .allocator = this. + }; + + var parse_result = this.bundler.parse(parse_options, null); + + // necessary because we don't run the linker + if (parse_result) |*res| { + for (res.ast.import_records.slice()) |*import| { + if (import.kind.isCommonJS()) { + import.do_commonjs_transform_in_printer = true; + import.module_id = @truncate(u32, std.hash.Wyhash.hash(0, import.path.pretty)); + } + } + } + + return parse_result; +} + +pub fn scan( + this: *Transpiler, + globalThis: *JSC.JSGlobalObject, + callframe: *JSC.CallFrame, +) callconv(.C) JSC.JSValue { + JSC.markBinding(@src()); + const arguments = callframe.arguments(3); + var args = JSC.Node.ArgumentsSlice.init(globalThis.bunVM(), arguments.ptr[0..arguments.len]); + defer args.arena.deinit(); + const code_arg = args.next() orelse { + globalThis.throwInvalidArgumentType("scan", "code", "string or Uint8Array"); + return .zero; + }; + + const code_holder = JSC.Node.SliceOrBuffer.fromJS(globalThis, args.arena.allocator(), code_arg) orelse { + globalThis.throwInvalidArgumentType("scan", "code", "string or Uint8Array"); + return .zero; + }; + + const code = code_holder.slice(); + args.eat(); + var exception_ref = [_]JSC.C.JSValueRef{null}; + var exception: JSC.C.ExceptionRef = &exception_ref; + + const loader: ?Loader = brk: { + if (args.next()) |arg| { + args.eat(); + break :brk Loader.fromJS(globalThis, arg, exception); + } + + break :brk null; + }; + + if (exception.* != null) { + globalThis.throwValue(JSC.JSValue.c(exception.*)); + return .zero; + } + + var arena = Mimalloc.Arena.init() catch unreachable; + var prev_allocator = this.bundler.allocator; + this.bundler.setAllocator(arena.allocator()); + var log = logger.Log.init(arena.backingAllocator()); + defer log.deinit(); + this.bundler.setLog(&log); + defer { + this.bundler.setLog(&this.transpiler_options.log); + this.bundler.setAllocator(prev_allocator); + arena.deinit(); + } + + defer { + JSAst.Stmt.Data.Store.reset(); + JSAst.Expr.Data.Store.reset(); + } + + const parse_result = getParseResult(this, arena.allocator(), code, loader, Bundler.MacroJSValueType.zero) orelse { + if ((this.bundler.log.warnings + this.bundler.log.errors) > 0) { + globalThis.throwValue(this.bundler.log.toJS(globalThis, globalThis.allocator(), "Parse error")); + return .zero; + } + + globalThis.throw("Failed to parse", .{}); + return .zero; + }; + + if ((this.bundler.log.warnings + this.bundler.log.errors) > 0) { + globalThis.throwValue(this.bundler.log.toJS(globalThis, globalThis.allocator(), "Parse error")); + return .zero; + } + + const exports_label = JSC.ZigString.static("exports"); + const imports_label = JSC.ZigString.static("imports"); + const named_imports_value = namedImportsToJS( + globalThis, + parse_result.ast.import_records.slice(), + exception, + ); + if (exception.* != null) { + globalThis.throwValue(JSC.JSValue.c(exception.*)); + return .zero; + } + + const named_exports_value = namedExportsToJS( + globalThis, + parse_result.ast.named_exports, + ); + return JSC.JSValue.createObject2(globalThis, imports_label, exports_label, named_imports_value, named_exports_value); +} + +// pub fn build( +// this: *Transpiler, +// ctx: js.JSContextRef, +// _: js.JSObjectRef, +// _: js.JSObjectRef, +// arguments: []const js.JSValueRef, +// exception: js.ExceptionRef, +// ) JSC.C.JSObjectRef {} + +pub fn transform( + this: *Transpiler, + globalThis: *JSC.JSGlobalObject, + callframe: *JSC.CallFrame, +) callconv(.C) JSC.JSValue { + JSC.markBinding(@src()); + var exception_ref = [_]JSC.C.JSValueRef{null}; + var exception: JSC.C.ExceptionRef = &exception_ref; + const arguments = callframe.arguments(3); + var args = JSC.Node.ArgumentsSlice.init(globalThis.bunVM(), arguments.ptr[0..arguments.len]); + defer args.arena.deinit(); + const code_arg = args.next() orelse { + globalThis.throwInvalidArgumentType("transform", "code", "string or Uint8Array"); + return .zero; + }; + + const code_holder = JSC.Node.StringOrBuffer.fromJS(globalThis, this.arena.allocator(), code_arg, exception) orelse { + globalThis.throwInvalidArgumentType("transform", "code", "string or Uint8Array"); + return .zero; + }; + + const code = code_holder.slice(); + args.eat(); + const loader: ?Loader = brk: { + if (args.next()) |arg| { + args.eat(); + break :brk Loader.fromJS(globalThis, arg, exception); + } + + break :brk null; + }; + + if (exception.* != null) { + globalThis.throwValue(JSC.JSValue.c(exception.*)); + return .zero; + } + + if (code_holder == .string) { + arguments.ptr[0].ensureStillAlive(); + } + + var task = TransformTask.create( + this, + if (code_holder == .string) arguments.ptr[0] else .zero, + globalThis, + ZigString.init(code), + loader orelse this.transpiler_options.default_loader, + ) catch { + globalThis.throw("Out of memory", .{}); + return .zero; + }; + task.schedule(); + return task.promise.value(); +} + +pub fn transformSync( + this: *Transpiler, + globalThis: *JSC.JSGlobalObject, + callframe: *JSC.CallFrame, +) callconv(.C) JSC.JSValue { + JSC.markBinding(@src()); + var exception_value = [_]JSC.C.JSValueRef{null}; + var exception: JSC.C.ExceptionRef = &exception_value; + const arguments = callframe.arguments(3); + + var args = JSC.Node.ArgumentsSlice.init(globalThis.bunVM(), arguments.ptr[0..arguments.len]); + defer args.arena.deinit(); + const code_arg = args.next() orelse { + globalThis.throwInvalidArgumentType("transformSync", "code", "string or Uint8Array"); + return .zero; + }; + + var arena = Mimalloc.Arena.init() catch unreachable; + defer arena.deinit(); + const code_holder = JSC.Node.StringOrBuffer.fromJS(globalThis, arena.allocator(), code_arg, exception) orelse { + globalThis.throwInvalidArgumentType("transformSync", "code", "string or Uint8Array"); + return .zero; + }; + + const code = code_holder.slice(); + arguments.ptr[0].ensureStillAlive(); + defer arguments.ptr[0].ensureStillAlive(); + + args.eat(); + var js_ctx_value: JSC.JSValue = JSC.JSValue.zero; + const loader: ?Loader = brk: { + if (args.next()) |arg| { + args.eat(); + if (arg.isNumber() or arg.isString()) { + break :brk Loader.fromJS(globalThis, arg, exception); + } + + if (arg.isObject()) { + js_ctx_value = arg; + break :brk null; + } + } + + break :brk null; + }; + + if (args.nextEat()) |arg| { + if (arg.isObject()) { + js_ctx_value = arg; + } else { + globalThis.throwInvalidArgumentType("transformSync", "context", "object or loader"); + return .zero; + } + } + if (!js_ctx_value.isEmpty()) { + js_ctx_value.ensureStillAlive(); + } + + defer { + if (!js_ctx_value.isEmpty()) { + js_ctx_value.ensureStillAlive(); + } + } + + if (exception.* != null) { + globalThis.throwValue(JSC.JSValue.c(exception.*)); + return .zero; + } + + JSAst.Stmt.Data.Store.reset(); + JSAst.Expr.Data.Store.reset(); + defer { + JSAst.Stmt.Data.Store.reset(); + JSAst.Expr.Data.Store.reset(); + } + + var prev_bundler = this.bundler; + this.bundler.setAllocator(arena.allocator()); + this.bundler.macro_context = null; + var log = logger.Log.init(arena.backingAllocator()); + this.bundler.setLog(&log); + + defer { + this.bundler = prev_bundler; + } + var parse_result = getParseResult( + this, + arena.allocator(), + code, + loader, + if (comptime JSC.is_bindgen) Bundler.MacroJSValueType.zero else js_ctx_value, + ) orelse { + if ((this.bundler.log.warnings + this.bundler.log.errors) > 0) { + globalThis.throwValue(this.bundler.log.toJS(globalThis, globalThis.allocator(), "Parse error")); + return .zero; + } + + globalThis.throw("Failed to parse code", .{}); + return .zero; + }; + + if ((this.bundler.log.warnings + this.bundler.log.errors) > 0) { + globalThis.throwValue(this.bundler.log.toJS(globalThis, globalThis.allocator(), "Parse error")); + return .zero; + } + + var buffer_writer = this.buffer_writer orelse brk: { + var writer = JSPrinter.BufferWriter.init(arena.backingAllocator()) catch { + globalThis.throw("Failed to create BufferWriter", .{}); + return .zero; + }; + + writer.buffer.growIfNeeded(code.len) catch unreachable; + writer.buffer.list.expandToCapacity(); + break :brk writer; + }; + + defer { + this.buffer_writer = buffer_writer; + } + + buffer_writer.reset(); + var printer = JSPrinter.BufferPrinter.init(buffer_writer); + _ = this.bundler.print(parse_result, @TypeOf(&printer), &printer, .esm_ascii) catch |err| { + globalThis.throwError(err, "Failed to print code"); + return .zero; + }; + + // TODO: benchmark if pooling this way is faster or moving is faster + buffer_writer = printer.ctx; + var out = JSC.ZigString.init(buffer_writer.written); + out.setOutputEncoding(); + + return out.toValueGC(globalThis); +} + +fn namedExportsToJS(global: *JSGlobalObject, named_exports: JSAst.Ast.NamedExports) JSC.JSValue { + if (named_exports.count() == 0) + return JSC.JSValue.fromRef(JSC.C.JSObjectMakeArray(global, 0, null, null)); + + var named_exports_iter = named_exports.iterator(); + var stack_fallback = std.heap.stackFallback(@sizeOf(JSC.ZigString) * 32, getAllocator(global)); + var allocator = stack_fallback.get(); + var names = allocator.alloc( + JSC.ZigString, + named_exports.count(), + ) catch unreachable; + defer allocator.free(names); + var i: usize = 0; + while (named_exports_iter.next()) |entry| { + names[i] = JSC.ZigString.init(entry.key_ptr.*); + i += 1; + } + JSC.ZigString.sortAsc(names[0..i]); + return JSC.JSValue.createStringArray(global, names.ptr, names.len, true); +} + +const ImportRecord = @import("../../import_record.zig").ImportRecord; + +fn namedImportsToJS( + global: *JSGlobalObject, + import_records: []const ImportRecord, + exception: JSC.C.ExceptionRef, +) JSC.JSValue { + var stack_fallback = std.heap.stackFallback(@sizeOf(JSC.C.JSObjectRef) * 32, getAllocator(global)); + var allocator = stack_fallback.get(); + + var i: usize = 0; + const path_label = JSC.ZigString.static("path"); + const kind_label = JSC.ZigString.static("kind"); + var array_items = allocator.alloc( + JSC.C.JSValueRef, + import_records.len, + ) catch unreachable; + defer allocator.free(array_items); + + for (import_records) |record| { + if (record.is_internal) continue; + + const path = JSC.ZigString.init(record.path.text).toValueGC(global); + const kind = JSC.ZigString.init(record.kind.label()).toValue(global); + array_items[i] = JSC.JSValue.createObject2(global, path_label, kind_label, path, kind).asObjectRef(); + i += 1; + } + + return JSC.JSValue.fromRef(JSC.C.JSObjectMakeArray(global, i, array_items.ptr, exception)); +} + +pub fn scanImports( + this: *Transpiler, + globalThis: *JSC.JSGlobalObject, + callframe: *JSC.CallFrame, +) callconv(.C) JSC.JSValue { + const arguments = callframe.arguments(2); + var exception_val = [_]JSC.C.JSValueRef{null}; + var exception: JSC.C.ExceptionRef = &exception_val; + var args = JSC.Node.ArgumentsSlice.init(globalThis.bunVM(), arguments.ptr[0..arguments.len]); + const code_arg = args.next() orelse { + globalThis.throwInvalidArgumentType("scanImports", "code", "string or Uint8Array"); + return .zero; + }; + + const code_holder = JSC.Node.StringOrBuffer.fromJS(globalThis, args.arena.allocator(), code_arg, exception) orelse { + if (exception.* == null) { + globalThis.throwInvalidArgumentType("scanImports", "code", "string or Uint8Array"); + } else { + globalThis.throwValue(JSC.JSValue.c(exception.*)); + } + + return .zero; + }; + args.eat(); + const code = code_holder.slice(); + + var loader: Loader = this.transpiler_options.default_loader; + if (args.next()) |arg| { + if (Loader.fromJS(globalThis, arg, exception)) |_loader| { + loader = _loader; + } + args.eat(); + } + + if (!loader.isJavaScriptLike()) { + globalThis.throwInvalidArguments("Only JavaScript-like files support this fast path", .{}); + return .zero; + } + + if (exception.* != null) { + globalThis.throwValue(JSC.JSValue.c(exception.*)); + return .zero; + } + + var arena = Mimalloc.Arena.init() catch unreachable; + var prev_allocator = this.bundler.allocator; + this.bundler.setAllocator(arena.allocator()); + var log = logger.Log.init(arena.backingAllocator()); + defer log.deinit(); + this.bundler.setLog(&log); + defer { + this.bundler.setLog(&this.transpiler_options.log); + this.bundler.setAllocator(prev_allocator); + arena.deinit(); + } + + const source = logger.Source.initPathString(loader.stdinName(), code); + var bundler = &this.bundler; + const jsx = if (this.transpiler_options.tsconfig != null) + this.transpiler_options.tsconfig.?.mergeJSX(this.bundler.options.jsx) + else + this.bundler.options.jsx; + + var opts = JSParser.Parser.Options.init(jsx, loader); + if (this.bundler.macro_context == null) { + this.bundler.macro_context = JSAst.Macro.MacroContext.init(&this.bundler); + } + opts.macro_context = &this.bundler.macro_context.?; + + JSAst.Stmt.Data.Store.reset(); + JSAst.Expr.Data.Store.reset(); + + defer { + JSAst.Stmt.Data.Store.reset(); + JSAst.Expr.Data.Store.reset(); + } + + bundler.resolver.caches.js.scan( + bundler.allocator, + &this.scan_pass_result, + opts, + bundler.options.define, + &log, + &source, + ) catch |err| { + defer this.scan_pass_result.reset(); + if ((log.warnings + log.errors) > 0) { + globalThis.throwValue(log.toJS(globalThis, globalThis.allocator(), "Failed to scan imports")); + return .zero; + } + + globalThis.throwError(err, "Failed to scan imports"); + return .zero; + }; + + defer this.scan_pass_result.reset(); + + if ((log.warnings + log.errors) > 0) { + globalThis.throwValue(log.toJS(globalThis, globalThis.allocator(), "Failed to scan imports")); + return .zero; + } + + const named_imports_value = namedImportsToJS( + globalThis, + this.scan_pass_result.import_records.items, + exception, + ); + if (exception.* != null) { + globalThis.throwValue(JSC.JSValue.c(exception.*)); + return .zero; + } + return named_imports_value; +} diff --git a/src/bun.js/api/bun.zig b/src/bun.js/api/bun.zig index 5dc3abcf9..7c6473727 100644 --- a/src/bun.js/api/bun.zig +++ b/src/bun.js/api/bun.zig @@ -71,7 +71,8 @@ const VM = @import("bun").JSC.VM; const JSFunction = @import("bun").JSC.JSFunction; const Config = @import("../config.zig"); const URL = @import("../../url.zig").URL; -const Transpiler = @import("./transpiler.zig"); +const Transpiler = bun.JSC.API.JSTranspiler; +const JSBundler = bun.JSC.API.JSBundler; const VirtualMachine = JSC.VirtualMachine; const IOTask = JSC.IOTask; const zlib = @import("../../zlib.zig"); @@ -1319,6 +1320,9 @@ pub const Class = NewClass( .Transpiler = .{ .get = getTranspilerConstructor, }, + .Bundler = .{ + .get = getBundlerConstructor, + }, .hash = .{ .get = getHashObject, }, @@ -2339,6 +2343,16 @@ pub fn mmapFile( }.x, @intToPtr(?*anyopaque, map.len), exception); } +pub fn getBundlerConstructor( + _: void, + ctx: js.JSContextRef, + _: js.JSValueRef, + _: js.JSStringRef, + _: js.ExceptionRef, +) js.JSValueRef { + return JSC.API.JSBundler.getConstructor(ctx).asObjectRef(); +} + pub fn getTranspilerConstructor( _: void, ctx: js.JSContextRef, @@ -2346,7 +2360,7 @@ pub fn getTranspilerConstructor( _: js.JSStringRef, _: js.ExceptionRef, ) js.JSValueRef { - return JSC.API.Bun.Transpiler.getConstructor(ctx).asObjectRef(); + return JSC.API.JSTranspiler.getConstructor(ctx).asObjectRef(); } pub fn getFileSystemRouter( diff --git a/src/bun.js/api/ffi.zig b/src/bun.js/api/ffi.zig index 89192dc85..6e2ad4242 100644 --- a/src/bun.js/api/ffi.zig +++ b/src/bun.js/api/ffi.zig @@ -71,7 +71,6 @@ const VM = @import("bun").JSC.VM; const JSFunction = @import("bun").JSC.JSFunction; const Config = @import("../config.zig"); const URL = @import("../../url.zig").URL; -const Transpiler = @import("./transpiler.zig"); const VirtualMachine = JSC.VirtualMachine; const IOTask = JSC.IOTask; const ComptimeStringMap = @import("../../comptime_string_map.zig").ComptimeStringMap; diff --git a/src/bun.js/api/server.zig b/src/bun.js/api/server.zig index 11237844a..bc83a021b 100644 --- a/src/bun.js/api/server.zig +++ b/src/bun.js/api/server.zig @@ -70,7 +70,6 @@ const VM = @import("bun").JSC.VM; const JSFunction = @import("bun").JSC.JSFunction; const Config = @import("../config.zig"); const URL = @import("../../url.zig").URL; -const Transpiler = @import("./transpiler.zig"); const VirtualMachine = JSC.VirtualMachine; const IOTask = JSC.IOTask; const is_bindgen = JSC.is_bindgen; @@ -5173,4 +5172,11 @@ pub const SSLServer = NewServer(true, false); pub const DebugServer = NewServer(false, true); pub const DebugSSLServer = NewServer(true, true); +pub const AnyServer = union(enum) { + Server: *Server, + SSLServer: *SSLServer, + DebugServer: *DebugServer, + DebugSSLServer: *DebugSSLServer, +}; + const welcome_page_html_gz = @embedFile("welcome-page.html.gz"); diff --git a/src/bun.js/api/transpiler.classes.ts b/src/bun.js/api/transpiler.classes.ts deleted file mode 100644 index f016a1844..000000000 --- a/src/bun.js/api/transpiler.classes.ts +++ /dev/null @@ -1,41 +0,0 @@ -import { define } from "../scripts/class-definitions"; - -export default [ - define({ - name: "Transpiler", - construct: true, - finalize: true, - hasPendingActivity: false, - configurable: false, - klass: {}, - JSType: "0b11101110", - proto: { - scanImports: { - fn: "scanImports", - length: 2, - }, - scan: { - fn: "scan", - length: 2, - }, - transform: { - fn: "transform", - length: 2, - }, - transformSync: { - fn: "transformSync", - length: 2, - }, - }, - custom: { - onLoadPlugins: { - extraHeaderIncludes: ["BunPlugin.h"], - impl: "JSTranspiler+BunPlugin-impl.h", - type: `WTF::Vector>`, - }, - onResolvePlugins: { - type: `WTF::Vector>`, - }, - }, - }), -]; diff --git a/src/bun.js/api/transpiler.zig b/src/bun.js/api/transpiler.zig deleted file mode 100644 index 8562a3204..000000000 --- a/src/bun.js/api/transpiler.zig +++ /dev/null @@ -1,1305 +0,0 @@ -const std = @import("std"); -const Api = @import("../../api/schema.zig").Api; -const http = @import("../../http.zig"); -const JavaScript = @import("../javascript.zig"); -const QueryStringMap = @import("../../url.zig").QueryStringMap; -const CombinedScanner = @import("../../url.zig").CombinedScanner; -const bun = @import("bun"); -const string = bun.string; -const JSC = @import("bun").JSC; -const js = JSC.C; -const WebCore = @import("../webcore/response.zig"); -const Bundler = bun.bundler; -const options = @import("../../options.zig"); -const VirtualMachine = JavaScript.VirtualMachine; -const ScriptSrcStream = std.io.FixedBufferStream([]u8); -const ZigString = JSC.ZigString; -const Fs = @import("../../fs.zig"); -const Base = @import("../base.zig"); -const getAllocator = Base.getAllocator; -const JSObject = JSC.JSObject; -const JSError = Base.JSError; -const JSValue = JSC.JSValue; -const JSGlobalObject = JSC.JSGlobalObject; -const strings = @import("bun").strings; -const NewClass = Base.NewClass; -const To = Base.To; -const Request = WebCore.Request; - -const FetchEvent = WebCore.FetchEvent; -const MacroMap = @import("../../resolver/package_json.zig").MacroMap; -const TSConfigJSON = @import("../../resolver/tsconfig_json.zig").TSConfigJSON; -const PackageJSON = @import("../../resolver/package_json.zig").PackageJSON; -const logger = @import("bun").logger; -const Loader = options.Loader; -const Platform = options.Platform; -const JSAst = bun.JSAst; -const Transpiler = @This(); -const JSParser = bun.js_parser; -const JSPrinter = bun.js_printer; -const ScanPassResult = JSParser.ScanPassResult; -const Mimalloc = @import("../../mimalloc_arena.zig"); -const Runtime = @import("../../runtime.zig").Runtime; -const JSLexer = bun.js_lexer; -const Expr = JSAst.Expr; - -pub usingnamespace JSC.Codegen.JSTranspiler; - -bundler: Bundler.Bundler, -arena: std.heap.ArenaAllocator, -transpiler_options: TranspilerOptions, -scan_pass_result: ScanPassResult, -buffer_writer: ?JSPrinter.BufferWriter = null, - -const default_transform_options: Api.TransformOptions = brk: { - var opts = std.mem.zeroes(Api.TransformOptions); - opts.disable_hmr = true; - opts.platform = Api.Platform.browser; - opts.serve = false; - break :brk opts; -}; - -const TranspilerOptions = struct { - transform: Api.TransformOptions = default_transform_options, - default_loader: options.Loader = options.Loader.jsx, - macro_map: MacroMap = MacroMap{}, - tsconfig: ?*TSConfigJSON = null, - tsconfig_buf: []const u8 = "", - macros_buf: []const u8 = "", - log: logger.Log, - runtime: Runtime.Features = Runtime.Features{ .top_level_await = true }, - tree_shaking: bool = false, - trim_unused_imports: ?bool = null, - inlining: bool = false, - - minify_whitespace: bool = false, - minify_identifiers: bool = false, -}; - -// Mimalloc gets unstable if we try to move this to a different thread -// threadlocal var transform_buffer: bun.MutableString = undefined; -// threadlocal var transform_buffer_loaded: bool = false; - -// This is going to be hard to not leak -pub const TransformTask = struct { - input_code: ZigString = ZigString.init(""), - protected_input_value: JSC.JSValue = @intToEnum(JSC.JSValue, 0), - output_code: ZigString = ZigString.init(""), - bundler: Bundler.Bundler = undefined, - log: logger.Log, - err: ?anyerror = null, - macro_map: MacroMap = MacroMap{}, - tsconfig: ?*TSConfigJSON = null, - loader: Loader, - global: *JSGlobalObject, - replace_exports: Runtime.Features.ReplaceableExport.Map = .{}, - - pub const AsyncTransformTask = JSC.ConcurrentPromiseTask(TransformTask); - pub const AsyncTransformEventLoopTask = AsyncTransformTask.EventLoopTask; - - pub fn create(transpiler: *Transpiler, protected_input_value: JSC.JSValue, globalThis: *JSGlobalObject, input_code: ZigString, loader: Loader) !*AsyncTransformTask { - var transform_task = try bun.default_allocator.create(TransformTask); - transform_task.* = .{ - .input_code = input_code, - .protected_input_value = protected_input_value, - .bundler = undefined, - .global = globalThis, - .macro_map = transpiler.transpiler_options.macro_map, - .tsconfig = transpiler.transpiler_options.tsconfig, - .log = logger.Log.init(bun.default_allocator), - .loader = loader, - .replace_exports = transpiler.transpiler_options.runtime.replace_exports, - }; - transform_task.bundler = transpiler.bundler; - transform_task.bundler.linker.resolver = &transform_task.bundler.resolver; - - transform_task.bundler.setLog(&transform_task.log); - transform_task.bundler.setAllocator(bun.default_allocator); - return try AsyncTransformTask.createOnJSThread(bun.default_allocator, globalThis, transform_task); - } - - pub fn run(this: *TransformTask) void { - const name = this.loader.stdinName(); - const source = logger.Source.initPathString(name, this.input_code.slice()); - - JSAst.Stmt.Data.Store.create(bun.default_allocator); - JSAst.Expr.Data.Store.create(bun.default_allocator); - - var arena = Mimalloc.Arena.init() catch unreachable; - - const allocator = arena.allocator(); - - defer { - JSAst.Stmt.Data.Store.reset(); - JSAst.Expr.Data.Store.reset(); - arena.deinit(); - } - - this.bundler.setAllocator(allocator); - const jsx = if (this.tsconfig != null) - this.tsconfig.?.mergeJSX(this.bundler.options.jsx) - else - this.bundler.options.jsx; - - const parse_options = Bundler.Bundler.ParseOptions{ - .allocator = allocator, - .macro_remappings = this.macro_map, - .dirname_fd = 0, - .file_descriptor = null, - .loader = this.loader, - .jsx = jsx, - .path = source.path, - .virtual_source = &source, - .replace_exports = this.replace_exports, - // .allocator = this. - }; - - const parse_result = this.bundler.parse(parse_options, null) orelse { - this.err = error.ParseError; - return; - }; - - if (parse_result.empty) { - this.output_code = ZigString.init(""); - return; - } - - var global_allocator = arena.backingAllocator(); - var buffer_writer = JSPrinter.BufferWriter.init(global_allocator) catch |err| { - this.err = err; - return; - }; - buffer_writer.buffer.list.ensureTotalCapacity(global_allocator, 512) catch unreachable; - buffer_writer.reset(); - - // defer { - // transform_buffer = buffer_writer.buffer; - // } - - var printer = JSPrinter.BufferPrinter.init(buffer_writer); - const printed = this.bundler.print(parse_result, @TypeOf(&printer), &printer, .esm_ascii) catch |err| { - this.err = err; - return; - }; - - if (printed > 0) { - buffer_writer = printer.ctx; - buffer_writer.buffer.list.items = buffer_writer.written; - - var output = JSC.ZigString.init(buffer_writer.written); - output.mark(); - this.output_code = output; - } else { - this.output_code = ZigString.init(""); - } - } - - pub fn then(this: *TransformTask, promise: *JSC.JSPromise) void { - if (this.log.hasAny() or this.err != null) { - const error_value: JSValue = brk: { - if (this.err) |err| { - if (!this.log.hasAny()) { - break :brk JSC.JSValue.fromRef(JSC.BuildError.create( - this.global, - bun.default_allocator, - logger.Msg{ - .data = logger.Data{ .text = bun.asByteSlice(@errorName(err)) }, - }, - )); - } - } - - break :brk this.log.toJS(this.global, bun.default_allocator, "Transform failed"); - }; - - promise.reject(this.global, error_value); - return; - } - - finish(this.output_code, this.global, promise); - - if (@enumToInt(this.protected_input_value) != 0) { - this.protected_input_value = @intToEnum(JSC.JSValue, 0); - } - this.deinit(); - } - - noinline fn finish(code: ZigString, global: *JSGlobalObject, promise: *JSC.JSPromise) void { - promise.resolve(global, code.toValueGC(global)); - } - - pub fn deinit(this: *TransformTask) void { - var should_cleanup = false; - defer if (should_cleanup) bun.Global.mimalloc_cleanup(false); - - this.log.deinit(); - if (this.input_code.isGloballyAllocated()) { - this.input_code.deinitGlobal(); - } - - if (this.output_code.isGloballyAllocated()) { - should_cleanup = this.output_code.len > 512_000; - this.output_code.deinitGlobal(); - } - - bun.default_allocator.destroy(this); - } -}; - -fn exportReplacementValue(value: JSValue, globalThis: *JSGlobalObject) ?JSAst.Expr { - if (value.isBoolean()) { - return Expr{ - .data = .{ - .e_boolean = .{ - .value = value.toBoolean(), - }, - }, - .loc = logger.Loc.Empty, - }; - } - - if (value.isNumber()) { - return Expr{ - .data = .{ - .e_number = .{ .value = value.asNumber() }, - }, - .loc = logger.Loc.Empty, - }; - } - - if (value.isNull()) { - return Expr{ - .data = .{ - .e_null = .{}, - }, - .loc = logger.Loc.Empty, - }; - } - - if (value.isUndefined()) { - return Expr{ - .data = .{ - .e_undefined = .{}, - }, - .loc = logger.Loc.Empty, - }; - } - - if (value.isString()) { - var str = JSAst.E.String{ - .data = std.fmt.allocPrint(bun.default_allocator, "{}", .{value.getZigString(globalThis)}) catch unreachable, - }; - var out = bun.default_allocator.create(JSAst.E.String) catch unreachable; - out.* = str; - return Expr{ - .data = .{ - .e_string = out, - }, - .loc = logger.Loc.Empty, - }; - } - - return null; -} - -fn transformOptionsFromJSC(globalObject: JSC.C.JSContextRef, temp_allocator: std.mem.Allocator, args: *JSC.Node.ArgumentsSlice, exception: JSC.C.ExceptionRef) !TranspilerOptions { - var globalThis = globalObject; - const object = args.next() orelse return TranspilerOptions{ .log = logger.Log.init(temp_allocator) }; - if (object.isUndefinedOrNull()) return TranspilerOptions{ .log = logger.Log.init(temp_allocator) }; - - args.eat(); - var allocator = args.arena.allocator(); - - var transpiler = TranspilerOptions{ - .default_loader = .jsx, - .transform = default_transform_options, - .log = logger.Log.init(allocator), - }; - transpiler.log.level = .warn; - - if (!object.isObject()) { - JSC.throwInvalidArguments("Expected an object", .{}, globalObject, exception); - return transpiler; - } - - if (object.getIfPropertyExists(globalObject, "define")) |define| { - define: { - if (define.isUndefinedOrNull()) { - break :define; - } - - if (!define.isObject()) { - JSC.throwInvalidArguments("define must be an object", .{}, globalObject, exception); - return transpiler; - } - - var define_iter = JSC.JSPropertyIterator(.{ - .skip_empty_name = true, - - .include_value = true, - }).init(globalThis, define.asObjectRef()); - defer define_iter.deinit(); - - // cannot be a temporary because it may be loaded on different threads. - var map_entries = allocator.alloc([]u8, define_iter.len * 2) catch unreachable; - var names = map_entries[0..define_iter.len]; - - var values = map_entries[define_iter.len..]; - - while (define_iter.next()) |prop| { - const property_value = define_iter.value; - const value_type = property_value.jsType(); - - if (!value_type.isStringLike()) { - JSC.throwInvalidArguments("define \"{s}\" must be a JSON string", .{prop}, globalObject, exception); - return transpiler; - } - - names[define_iter.i] = prop.toOwnedSlice(allocator) catch unreachable; - var val = JSC.ZigString.init(""); - property_value.toZigString(&val, globalThis); - if (val.len == 0) { - val = JSC.ZigString.init("\"\""); - } - values[define_iter.i] = std.fmt.allocPrint(allocator, "{}", .{val}) catch unreachable; - } - - transpiler.transform.define = Api.StringMap{ - .keys = names, - .values = values, - }; - } - } - - if (object.get(globalThis, "external")) |external| { - external: { - if (external.isUndefinedOrNull()) break :external; - - const toplevel_type = external.jsType(); - if (toplevel_type.isStringLike()) { - var zig_str = JSC.ZigString.init(""); - external.toZigString(&zig_str, globalThis); - if (zig_str.len == 0) break :external; - var single_external = allocator.alloc(string, 1) catch unreachable; - single_external[0] = std.fmt.allocPrint(allocator, "{}", .{external}) catch unreachable; - transpiler.transform.external = single_external; - } else if (toplevel_type.isArray()) { - const count = external.getLengthOfArray(globalThis); - if (count == 0) break :external; - - var externals = allocator.alloc(string, count) catch unreachable; - var iter = external.arrayIterator(globalThis); - var i: usize = 0; - while (iter.next()) |entry| { - if (!entry.jsType().isStringLike()) { - JSC.throwInvalidArguments("external must be a string or string[]", .{}, globalObject, exception); - return transpiler; - } - - var zig_str = JSC.ZigString.init(""); - entry.toZigString(&zig_str, globalThis); - if (zig_str.len == 0) continue; - externals[i] = std.fmt.allocPrint(allocator, "{}", .{external}) catch unreachable; - i += 1; - } - - transpiler.transform.external = externals[0..i]; - } else { - JSC.throwInvalidArguments("external must be a string or string[]", .{}, globalObject, exception); - return transpiler; - } - } - } - - if (object.get(globalThis, "loader")) |loader| { - if (Loader.fromJS(globalThis, loader, exception)) |resolved| { - if (!resolved.isJavaScriptLike()) { - JSC.throwInvalidArguments("only JavaScript-like loaders supported for now", .{}, globalObject, exception); - return transpiler; - } - - transpiler.default_loader = resolved; - } - - if (exception.* != null) { - return transpiler; - } - } - - if (object.get(globalThis, "platform")) |platform| { - if (Platform.fromJS(globalThis, platform, exception)) |resolved| { - transpiler.transform.platform = resolved.toAPI(); - } - - if (exception.* != null) { - return transpiler; - } - } - - if (object.get(globalThis, "tsconfig")) |tsconfig| { - tsconfig: { - if (tsconfig.isUndefinedOrNull()) break :tsconfig; - const kind = tsconfig.jsType(); - var out = JSC.ZigString.init(""); - - if (kind.isArray()) { - JSC.throwInvalidArguments("tsconfig must be a string or object", .{}, globalObject, exception); - return transpiler; - } - - if (!kind.isStringLike()) { - tsconfig.jsonStringify(globalThis, 0, &out); - } else { - tsconfig.toZigString(&out, globalThis); - } - - if (out.len == 0) break :tsconfig; - transpiler.tsconfig_buf = std.fmt.allocPrint(allocator, "{}", .{out}) catch unreachable; - - // TODO: JSC -> Ast conversion - if (TSConfigJSON.parse( - allocator, - &transpiler.log, - logger.Source.initPathString("tsconfig.json", transpiler.tsconfig_buf), - &VirtualMachine.get().bundler.resolver.caches.json, - true, - ) catch null) |parsed_tsconfig| { - transpiler.tsconfig = parsed_tsconfig; - } - } - } - - transpiler.runtime.allow_runtime = false; - transpiler.runtime.dynamic_require = switch (transpiler.transform.platform orelse .browser) { - .bun, .bun_macro => true, - else => false, - }; - - if (object.getIfPropertyExists(globalThis, "macro")) |macros| { - macros: { - if (macros.isUndefinedOrNull()) break :macros; - const kind = macros.jsType(); - const is_object = kind.isObject(); - if (!(kind.isStringLike() or is_object)) { - JSC.throwInvalidArguments("macro must be an object", .{}, globalObject, exception); - return transpiler; - } - - var out: ZigString = ZigString.init(""); - // TODO: write a converter between JSC types and Bun AST types - if (is_object) { - macros.jsonStringify(globalThis, 0, &out); - } else { - macros.toZigString(&out, globalThis); - } - - if (out.len == 0) break :macros; - transpiler.macros_buf = std.fmt.allocPrint(allocator, "{}", .{out}) catch unreachable; - const source = logger.Source.initPathString("macros.json", transpiler.macros_buf); - const json = (VirtualMachine.get().bundler.resolver.caches.json.parseJSON( - &transpiler.log, - source, - allocator, - ) catch null) orelse break :macros; - transpiler.macro_map = PackageJSON.parseMacrosJSON(allocator, json, &transpiler.log, &source); - } - } - - if (object.get(globalThis, "autoImportJSX")) |flag| { - transpiler.runtime.auto_import_jsx = flag.toBoolean(); - } - - if (object.get(globalThis, "allowBunRuntime")) |flag| { - transpiler.runtime.allow_runtime = flag.toBoolean(); - } - - if (object.get(globalThis, "jsxOptimizationInline")) |flag| { - transpiler.runtime.jsx_optimization_inline = flag.toBoolean(); - } - - if (object.get(globalThis, "jsxOptimizationHoist")) |flag| { - transpiler.runtime.jsx_optimization_hoist = flag.toBoolean(); - - if (!transpiler.runtime.jsx_optimization_inline and transpiler.runtime.jsx_optimization_hoist) { - JSC.throwInvalidArguments("jsxOptimizationHoist requires jsxOptimizationInline", .{}, globalObject, exception); - return transpiler; - } - } - - if (object.get(globalThis, "inline")) |flag| { - transpiler.runtime.inlining = flag.toBoolean(); - } - - if (object.get(globalThis, "minifyWhitespace")) |flag| { - transpiler.minify_whitespace = flag.toBoolean(); - } - - if (object.get(globalThis, "sourcemap")) |flag| { - if (flag.isBoolean() or flag.isUndefinedOrNull()) { - if (flag.toBoolean()) { - transpiler.transform.source_map = Api.SourceMapMode.external; - } else { - transpiler.transform.source_map = Api.SourceMapMode.inline_into_file; - } - } else { - var sourcemap = flag.toSlice(globalThis, allocator); - if (options.SourceMapOption.map.get(sourcemap.slice())) |source| { - transpiler.transform.source_map = source.toAPI(); - } else { - JSC.throwInvalidArguments("sourcemap must be one of \"inline\", \"external\", or \"none\"", .{}, globalObject, exception); - return transpiler; - } - } - } - - var tree_shaking: ?bool = null; - if (object.get(globalThis, "treeShaking")) |treeShaking| { - tree_shaking = treeShaking.toBoolean(); - } - - var trim_unused_imports: ?bool = null; - if (object.get(globalThis, "trimUnusedImports")) |trimUnusedImports| { - trim_unused_imports = trimUnusedImports.toBoolean(); - } - - if (object.getTruthy(globalThis, "exports")) |exports| { - if (!exports.isObject()) { - JSC.throwInvalidArguments("exports must be an object", .{}, globalObject, exception); - return transpiler; - } - - var replacements = Runtime.Features.ReplaceableExport.Map{}; - errdefer replacements.clearAndFree(bun.default_allocator); - - if (exports.getTruthy(globalThis, "eliminate")) |eliminate| { - if (!eliminate.jsType().isArray()) { - JSC.throwInvalidArguments("exports.eliminate must be an array", .{}, globalObject, exception); - return transpiler; - } - - var total_name_buf_len: u32 = 0; - var string_count: u32 = 0; - var iter = JSC.JSArrayIterator.init(eliminate, globalThis); - { - var length_iter = iter; - while (length_iter.next()) |value| { - if (value.isString()) { - const length = @truncate(u32, value.getLengthOfArray(globalThis)); - string_count += @as(u32, @boolToInt(length > 0)); - total_name_buf_len += length; - } - } - } - - if (total_name_buf_len > 0) { - var buf = try std.ArrayListUnmanaged(u8).initCapacity(bun.default_allocator, total_name_buf_len); - try replacements.ensureUnusedCapacity(bun.default_allocator, string_count); - { - var length_iter = iter; - while (length_iter.next()) |value| { - if (!value.isString()) continue; - var str = value.getZigString(globalThis); - if (str.len == 0) continue; - const name = std.fmt.bufPrint(buf.items.ptr[buf.items.len..buf.capacity], "{}", .{str}) catch { - JSC.throwInvalidArguments("Error reading exports.eliminate. TODO: utf-16", .{}, globalObject, exception); - return transpiler; - }; - buf.items.len += name.len; - if (name.len > 0) { - replacements.putAssumeCapacity(name, .{ .delete = {} }); - } - } - } - } - } - - if (exports.getTruthy(globalThis, "replace")) |replace| { - if (!replace.isObject()) { - JSC.throwInvalidArguments("replace must be an object", .{}, globalObject, exception); - return transpiler; - } - - var iter = JSC.JSPropertyIterator(.{ - .skip_empty_name = true, - .include_value = true, - }).init(globalThis, replace.asObjectRef()); - - if (iter.len > 0) { - errdefer iter.deinit(); - try replacements.ensureUnusedCapacity(bun.default_allocator, iter.len); - - // We cannot set the exception before `try` because it could be - // a double free with the `errdefer`. - defer if (exception.* != null) { - iter.deinit(); - for (replacements.keys()) |key| { - bun.default_allocator.free(bun.constStrToU8(key)); - } - replacements.clearAndFree(bun.default_allocator); - }; - - while (iter.next()) |key_| { - const value = iter.value; - if (value.isEmpty()) continue; - - var key = try key_.toOwnedSlice(bun.default_allocator); - - if (!JSLexer.isIdentifier(key)) { - JSC.throwInvalidArguments("\"{s}\" is not a valid ECMAScript identifier", .{key}, globalObject, exception); - bun.default_allocator.free(key); - return transpiler; - } - - var entry = replacements.getOrPutAssumeCapacity(key); - - if (exportReplacementValue(value, globalThis)) |expr| { - entry.value_ptr.* = .{ .replace = expr }; - continue; - } - - if (value.isObject() and value.getLengthOfArray(globalObject) == 2) { - const replacementValue = JSC.JSObject.getIndex(value, globalThis, 1); - if (exportReplacementValue(replacementValue, globalThis)) |to_replace| { - const replacementKey = JSC.JSObject.getIndex(value, globalThis, 0); - var slice = (try replacementKey.toSlice(globalThis, bun.default_allocator).cloneIfNeeded(bun.default_allocator)); - var replacement_name = slice.slice(); - - if (!JSLexer.isIdentifier(replacement_name)) { - JSC.throwInvalidArguments("\"{s}\" is not a valid ECMAScript identifier", .{replacement_name}, globalObject, exception); - slice.deinit(); - return transpiler; - } - - entry.value_ptr.* = .{ - .inject = .{ - .name = replacement_name, - .value = to_replace, - }, - }; - continue; - } - } - - JSC.throwInvalidArguments("exports.replace values can only be string, null, undefined, number or boolean", .{}, globalObject, exception); - return transpiler; - } - } - } - - tree_shaking = tree_shaking orelse (replacements.count() > 0); - transpiler.runtime.replace_exports = replacements; - } - - transpiler.tree_shaking = tree_shaking orelse false; - transpiler.trim_unused_imports = trim_unused_imports orelse transpiler.tree_shaking; - - return transpiler; -} - -pub fn constructor( - globalThis: *JSC.JSGlobalObject, - callframe: *JSC.CallFrame, -) callconv(.C) ?*Transpiler { - var temp = std.heap.ArenaAllocator.init(getAllocator(globalThis)); - const arguments = callframe.arguments(3); - var args = JSC.Node.ArgumentsSlice.init( - globalThis.bunVM(), - arguments.ptr[0..arguments.len], - ); - - defer temp.deinit(); - var exception_ref = [_]JSC.C.JSValueRef{null}; - var exception = &exception_ref[0]; - const transpiler_options: TranspilerOptions = if (arguments.len > 0) - transformOptionsFromJSC(globalThis, temp.allocator(), &args, exception) catch { - JSC.throwInvalidArguments("Failed to create transpiler", .{}, globalThis, exception); - return null; - } - else - TranspilerOptions{ .log = logger.Log.init(getAllocator(globalThis)) }; - - if (exception.* != null) { - globalThis.throwValue(JSC.JSValue.c(exception.*)); - return null; - } - - const allocator = getAllocator(globalThis); - - if ((transpiler_options.log.warnings + transpiler_options.log.errors) > 0) { - globalThis.throwValue( - transpiler_options.log.toJS(globalThis.ptr(), allocator, "Failed to create transpiler"), - ); - - return null; - } - - var log = allocator.create(logger.Log) catch unreachable; - log.* = transpiler_options.log; - var bundler = Bundler.Bundler.init( - allocator, - log, - transpiler_options.transform, - null, - JavaScript.VirtualMachine.get().bundler.env, - ) catch |err| { - if ((log.warnings + log.errors) > 0) { - globalThis.throwValue( - log.toJS(globalThis.ptr(), allocator, "Failed to create transpiler"), - ); - - return null; - } - - globalThis.throwError(err, "Error creating transpiler"); - return null; - }; - - bundler.configureLinkerWithAutoJSX(false); - bundler.options.env.behavior = .disable; - bundler.configureDefines() catch |err| { - if ((log.warnings + log.errors) > 0) { - globalThis.throwValue( - log.toJS(globalThis.ptr(), allocator, "Failed to load define"), - ); - - return null; - } - - globalThis.throwError(err, "Failed to load define"); - return null; - }; - - if (transpiler_options.macro_map.count() > 0) { - bundler.options.macro_remap = transpiler_options.macro_map; - } - - bundler.options.minify_whitespace = transpiler_options.minify_whitespace; - bundler.options.tree_shaking = transpiler_options.tree_shaking; - bundler.options.trim_unused_imports = transpiler_options.trim_unused_imports; - bundler.options.allow_runtime = transpiler_options.runtime.allow_runtime; - bundler.options.auto_import_jsx = transpiler_options.runtime.auto_import_jsx; - bundler.options.inlining = transpiler_options.runtime.inlining; - bundler.options.hot_module_reloading = transpiler_options.runtime.hot_module_reloading; - bundler.options.jsx.supports_fast_refresh = bundler.options.hot_module_reloading and - bundler.options.allow_runtime and transpiler_options.runtime.react_fast_refresh; - - var transpiler = allocator.create(Transpiler) catch unreachable; - transpiler.* = Transpiler{ - .transpiler_options = transpiler_options, - .bundler = bundler, - .arena = args.arena, - .scan_pass_result = ScanPassResult.init(allocator), - }; - - return transpiler; -} - -pub fn finalize( - this: *Transpiler, -) callconv(.C) void { - this.bundler.log.deinit(); - this.scan_pass_result.named_imports.deinit(); - this.scan_pass_result.import_records.deinit(); - this.scan_pass_result.used_symbols.deinit(); - if (this.buffer_writer != null) { - this.buffer_writer.?.buffer.deinit(); - } - - // bun.default_allocator.free(this.transpiler_options.tsconfig_buf); - // bun.default_allocator.free(this.transpiler_options.macros_buf); - this.arena.deinit(); - JSC.VirtualMachine.get().allocator.destroy(this); -} - -fn getParseResult(this: *Transpiler, allocator: std.mem.Allocator, code: []const u8, loader: ?Loader, macro_js_ctx: Bundler.MacroJSValueType) ?Bundler.ParseResult { - const name = this.transpiler_options.default_loader.stdinName(); - const source = logger.Source.initPathString(name, code); - - const jsx = if (this.transpiler_options.tsconfig != null) - this.transpiler_options.tsconfig.?.mergeJSX(this.bundler.options.jsx) - else - this.bundler.options.jsx; - - const parse_options = Bundler.Bundler.ParseOptions{ - .allocator = allocator, - .macro_remappings = this.transpiler_options.macro_map, - .dirname_fd = 0, - .file_descriptor = null, - .loader = loader orelse this.transpiler_options.default_loader, - .jsx = jsx, - .path = source.path, - .virtual_source = &source, - .replace_exports = this.transpiler_options.runtime.replace_exports, - .macro_js_ctx = macro_js_ctx, - // .allocator = this. - }; - - var parse_result = this.bundler.parse(parse_options, null); - - // necessary because we don't run the linker - if (parse_result) |*res| { - for (res.ast.import_records) |*import| { - if (import.kind.isCommonJS()) { - import.do_commonjs_transform_in_printer = true; - import.module_id = @truncate(u32, std.hash.Wyhash.hash(0, import.path.pretty)); - } - } - } - - return parse_result; -} - -pub fn scan( - this: *Transpiler, - globalThis: *JSC.JSGlobalObject, - callframe: *JSC.CallFrame, -) callconv(.C) JSC.JSValue { - JSC.markBinding(@src()); - const arguments = callframe.arguments(3); - var args = JSC.Node.ArgumentsSlice.init(globalThis.bunVM(), arguments.ptr[0..arguments.len]); - defer args.arena.deinit(); - const code_arg = args.next() orelse { - globalThis.throwInvalidArgumentType("scan", "code", "string or Uint8Array"); - return .zero; - }; - - const code_holder = JSC.Node.SliceOrBuffer.fromJS(globalThis, args.arena.allocator(), code_arg) orelse { - globalThis.throwInvalidArgumentType("scan", "code", "string or Uint8Array"); - return .zero; - }; - - const code = code_holder.slice(); - args.eat(); - var exception_ref = [_]JSC.C.JSValueRef{null}; - var exception: JSC.C.ExceptionRef = &exception_ref; - - const loader: ?Loader = brk: { - if (args.next()) |arg| { - args.eat(); - break :brk Loader.fromJS(globalThis, arg, exception); - } - - break :brk null; - }; - - if (exception.* != null) { - globalThis.throwValue(JSC.JSValue.c(exception.*)); - return .zero; - } - - var arena = Mimalloc.Arena.init() catch unreachable; - var prev_allocator = this.bundler.allocator; - this.bundler.setAllocator(arena.allocator()); - var log = logger.Log.init(arena.backingAllocator()); - defer log.deinit(); - this.bundler.setLog(&log); - defer { - this.bundler.setLog(&this.transpiler_options.log); - this.bundler.setAllocator(prev_allocator); - arena.deinit(); - } - - defer { - JSAst.Stmt.Data.Store.reset(); - JSAst.Expr.Data.Store.reset(); - } - - const parse_result = getParseResult(this, arena.allocator(), code, loader, Bundler.MacroJSValueType.zero) orelse { - if ((this.bundler.log.warnings + this.bundler.log.errors) > 0) { - globalThis.throwValue(this.bundler.log.toJS(globalThis, globalThis.allocator(), "Parse error")); - return .zero; - } - - globalThis.throw("Failed to parse", .{}); - return .zero; - }; - - if ((this.bundler.log.warnings + this.bundler.log.errors) > 0) { - globalThis.throwValue(this.bundler.log.toJS(globalThis, globalThis.allocator(), "Parse error")); - return .zero; - } - - const exports_label = JSC.ZigString.static("exports"); - const imports_label = JSC.ZigString.static("imports"); - const named_imports_value = namedImportsToJS( - globalThis, - parse_result.ast.import_records, - exception, - ); - if (exception.* != null) { - globalThis.throwValue(JSC.JSValue.c(exception.*)); - return .zero; - } - - const named_exports_value = namedExportsToJS( - globalThis, - parse_result.ast.named_exports, - ); - return JSC.JSValue.createObject2(globalThis, imports_label, exports_label, named_imports_value, named_exports_value); -} - -// pub fn build( -// this: *Transpiler, -// ctx: js.JSContextRef, -// _: js.JSObjectRef, -// _: js.JSObjectRef, -// arguments: []const js.JSValueRef, -// exception: js.ExceptionRef, -// ) JSC.C.JSObjectRef {} - -pub fn transform( - this: *Transpiler, - globalThis: *JSC.JSGlobalObject, - callframe: *JSC.CallFrame, -) callconv(.C) JSC.JSValue { - JSC.markBinding(@src()); - var exception_ref = [_]JSC.C.JSValueRef{null}; - var exception: JSC.C.ExceptionRef = &exception_ref; - const arguments = callframe.arguments(3); - var args = JSC.Node.ArgumentsSlice.init(globalThis.bunVM(), arguments.ptr[0..arguments.len]); - defer args.arena.deinit(); - const code_arg = args.next() orelse { - globalThis.throwInvalidArgumentType("transform", "code", "string or Uint8Array"); - return .zero; - }; - - const code_holder = JSC.Node.StringOrBuffer.fromJS(globalThis, this.arena.allocator(), code_arg, exception) orelse { - globalThis.throwInvalidArgumentType("transform", "code", "string or Uint8Array"); - return .zero; - }; - - const code = code_holder.slice(); - args.eat(); - const loader: ?Loader = brk: { - if (args.next()) |arg| { - args.eat(); - break :brk Loader.fromJS(globalThis, arg, exception); - } - - break :brk null; - }; - - if (exception.* != null) { - globalThis.throwValue(JSC.JSValue.c(exception.*)); - return .zero; - } - - if (code_holder == .string) { - arguments.ptr[0].ensureStillAlive(); - } - - var task = TransformTask.create( - this, - if (code_holder == .string) arguments.ptr[0] else .zero, - globalThis, - ZigString.init(code), - loader orelse this.transpiler_options.default_loader, - ) catch { - globalThis.throw("Out of memory", .{}); - return .zero; - }; - task.schedule(); - return task.promise.value(); -} - -pub fn transformSync( - this: *Transpiler, - globalThis: *JSC.JSGlobalObject, - callframe: *JSC.CallFrame, -) callconv(.C) JSC.JSValue { - JSC.markBinding(@src()); - var exception_value = [_]JSC.C.JSValueRef{null}; - var exception: JSC.C.ExceptionRef = &exception_value; - const arguments = callframe.arguments(3); - - var args = JSC.Node.ArgumentsSlice.init(globalThis.bunVM(), arguments.ptr[0..arguments.len]); - defer args.arena.deinit(); - const code_arg = args.next() orelse { - globalThis.throwInvalidArgumentType("transformSync", "code", "string or Uint8Array"); - return .zero; - }; - - var arena = Mimalloc.Arena.init() catch unreachable; - defer arena.deinit(); - const code_holder = JSC.Node.StringOrBuffer.fromJS(globalThis, arena.allocator(), code_arg, exception) orelse { - globalThis.throwInvalidArgumentType("transformSync", "code", "string or Uint8Array"); - return .zero; - }; - - const code = code_holder.slice(); - arguments.ptr[0].ensureStillAlive(); - defer arguments.ptr[0].ensureStillAlive(); - - args.eat(); - var js_ctx_value: JSC.JSValue = JSC.JSValue.zero; - const loader: ?Loader = brk: { - if (args.next()) |arg| { - args.eat(); - if (arg.isNumber() or arg.isString()) { - break :brk Loader.fromJS(globalThis, arg, exception); - } - - if (arg.isObject()) { - js_ctx_value = arg; - break :brk null; - } - } - - break :brk null; - }; - - if (args.nextEat()) |arg| { - if (arg.isObject()) { - js_ctx_value = arg; - } else { - globalThis.throwInvalidArgumentType("transformSync", "context", "object or loader"); - return .zero; - } - } - if (!js_ctx_value.isEmpty()) { - js_ctx_value.ensureStillAlive(); - } - - defer { - if (!js_ctx_value.isEmpty()) { - js_ctx_value.ensureStillAlive(); - } - } - - if (exception.* != null) { - globalThis.throwValue(JSC.JSValue.c(exception.*)); - return .zero; - } - - JSAst.Stmt.Data.Store.reset(); - JSAst.Expr.Data.Store.reset(); - defer { - JSAst.Stmt.Data.Store.reset(); - JSAst.Expr.Data.Store.reset(); - } - - var prev_bundler = this.bundler; - this.bundler.setAllocator(arena.allocator()); - this.bundler.macro_context = null; - var log = logger.Log.init(arena.backingAllocator()); - this.bundler.setLog(&log); - - defer { - this.bundler = prev_bundler; - } - var parse_result = getParseResult( - this, - arena.allocator(), - code, - loader, - if (comptime JSC.is_bindgen) Bundler.MacroJSValueType.zero else js_ctx_value, - ) orelse { - if ((this.bundler.log.warnings + this.bundler.log.errors) > 0) { - globalThis.throwValue(this.bundler.log.toJS(globalThis, globalThis.allocator(), "Parse error")); - return .zero; - } - - globalThis.throw("Failed to parse code", .{}); - return .zero; - }; - - if ((this.bundler.log.warnings + this.bundler.log.errors) > 0) { - globalThis.throwValue(this.bundler.log.toJS(globalThis, globalThis.allocator(), "Parse error")); - return .zero; - } - - var buffer_writer = this.buffer_writer orelse brk: { - var writer = JSPrinter.BufferWriter.init(arena.backingAllocator()) catch { - globalThis.throw("Failed to create BufferWriter", .{}); - return .zero; - }; - - writer.buffer.growIfNeeded(code.len) catch unreachable; - writer.buffer.list.expandToCapacity(); - break :brk writer; - }; - - defer { - this.buffer_writer = buffer_writer; - } - - buffer_writer.reset(); - var printer = JSPrinter.BufferPrinter.init(buffer_writer); - _ = this.bundler.print(parse_result, @TypeOf(&printer), &printer, .esm_ascii) catch |err| { - globalThis.throwError(err, "Failed to print code"); - return .zero; - }; - - // TODO: benchmark if pooling this way is faster or moving is faster - buffer_writer = printer.ctx; - var out = JSC.ZigString.init(buffer_writer.written); - out.setOutputEncoding(); - - return out.toValueGC(globalThis); -} - -fn namedExportsToJS(global: *JSGlobalObject, named_exports: JSAst.Ast.NamedExports) JSC.JSValue { - if (named_exports.count() == 0) - return JSC.JSValue.fromRef(JSC.C.JSObjectMakeArray(global, 0, null, null)); - - var named_exports_iter = named_exports.iterator(); - var stack_fallback = std.heap.stackFallback(@sizeOf(JSC.ZigString) * 32, getAllocator(global)); - var allocator = stack_fallback.get(); - var names = allocator.alloc( - JSC.ZigString, - named_exports.count(), - ) catch unreachable; - defer allocator.free(names); - var i: usize = 0; - while (named_exports_iter.next()) |entry| { - names[i] = JSC.ZigString.init(entry.key_ptr.*); - i += 1; - } - JSC.ZigString.sortAsc(names[0..i]); - return JSC.JSValue.createStringArray(global, names.ptr, names.len, true); -} - -const ImportRecord = @import("../../import_record.zig").ImportRecord; - -fn namedImportsToJS( - global: *JSGlobalObject, - import_records: []const ImportRecord, - exception: JSC.C.ExceptionRef, -) JSC.JSValue { - var stack_fallback = std.heap.stackFallback(@sizeOf(JSC.C.JSObjectRef) * 32, getAllocator(global)); - var allocator = stack_fallback.get(); - - var i: usize = 0; - const path_label = JSC.ZigString.static("path"); - const kind_label = JSC.ZigString.static("kind"); - var array_items = allocator.alloc( - JSC.C.JSValueRef, - import_records.len, - ) catch unreachable; - defer allocator.free(array_items); - - for (import_records) |record| { - if (record.is_internal) continue; - - const path = JSC.ZigString.init(record.path.text).toValueGC(global); - const kind = JSC.ZigString.init(record.kind.label()).toValue(global); - array_items[i] = JSC.JSValue.createObject2(global, path_label, kind_label, path, kind).asObjectRef(); - i += 1; - } - - return JSC.JSValue.fromRef(JSC.C.JSObjectMakeArray(global, i, array_items.ptr, exception)); -} - -pub fn scanImports( - this: *Transpiler, - globalThis: *JSC.JSGlobalObject, - callframe: *JSC.CallFrame, -) callconv(.C) JSC.JSValue { - const arguments = callframe.arguments(2); - var exception_val = [_]JSC.C.JSValueRef{null}; - var exception: JSC.C.ExceptionRef = &exception_val; - var args = JSC.Node.ArgumentsSlice.init(globalThis.bunVM(), arguments.ptr[0..arguments.len]); - const code_arg = args.next() orelse { - globalThis.throwInvalidArgumentType("scanImports", "code", "string or Uint8Array"); - return .zero; - }; - - const code_holder = JSC.Node.StringOrBuffer.fromJS(globalThis, args.arena.allocator(), code_arg, exception) orelse { - if (exception.* == null) { - globalThis.throwInvalidArgumentType("scanImports", "code", "string or Uint8Array"); - } else { - globalThis.throwValue(JSC.JSValue.c(exception.*)); - } - - return .zero; - }; - args.eat(); - const code = code_holder.slice(); - - var loader: Loader = this.transpiler_options.default_loader; - if (args.next()) |arg| { - if (Loader.fromJS(globalThis, arg, exception)) |_loader| { - loader = _loader; - } - args.eat(); - } - - if (!loader.isJavaScriptLike()) { - globalThis.throwInvalidArguments("Only JavaScript-like files support this fast path", .{}); - return .zero; - } - - if (exception.* != null) { - globalThis.throwValue(JSC.JSValue.c(exception.*)); - return .zero; - } - - var arena = Mimalloc.Arena.init() catch unreachable; - var prev_allocator = this.bundler.allocator; - this.bundler.setAllocator(arena.allocator()); - var log = logger.Log.init(arena.backingAllocator()); - defer log.deinit(); - this.bundler.setLog(&log); - defer { - this.bundler.setLog(&this.transpiler_options.log); - this.bundler.setAllocator(prev_allocator); - arena.deinit(); - } - - const source = logger.Source.initPathString(loader.stdinName(), code); - var bundler = &this.bundler; - const jsx = if (this.transpiler_options.tsconfig != null) - this.transpiler_options.tsconfig.?.mergeJSX(this.bundler.options.jsx) - else - this.bundler.options.jsx; - - var opts = JSParser.Parser.Options.init(jsx, loader); - if (this.bundler.macro_context == null) { - this.bundler.macro_context = JSAst.Macro.MacroContext.init(&this.bundler); - } - opts.macro_context = &this.bundler.macro_context.?; - - JSAst.Stmt.Data.Store.reset(); - JSAst.Expr.Data.Store.reset(); - - defer { - JSAst.Stmt.Data.Store.reset(); - JSAst.Expr.Data.Store.reset(); - } - - bundler.resolver.caches.js.scan( - bundler.allocator, - &this.scan_pass_result, - opts, - bundler.options.define, - &log, - &source, - ) catch |err| { - defer this.scan_pass_result.reset(); - if ((log.warnings + log.errors) > 0) { - globalThis.throwValue(log.toJS(globalThis, globalThis.allocator(), "Failed to scan imports")); - return .zero; - } - - globalThis.throwError(err, "Failed to scan imports"); - return .zero; - }; - - defer this.scan_pass_result.reset(); - - if ((log.warnings + log.errors) > 0) { - globalThis.throwValue(log.toJS(globalThis, globalThis.allocator(), "Failed to scan imports")); - return .zero; - } - - const named_imports_value = namedImportsToJS( - globalThis, - this.scan_pass_result.import_records.items, - exception, - ); - if (exception.* != null) { - globalThis.throwValue(JSC.JSValue.c(exception.*)); - return .zero; - } - return named_imports_value; -} -- cgit v1.2.3