Skip to content
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
219 changes: 219 additions & 0 deletions rfcs/static-require-bundle.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,219 @@
# RFC: Static require() for ServerLua

* Status: **DRAFT**

## Summary

Compile-time static `require()` resolution that bundles multiple modules into a single deployable unit. Client-side bundling
avoids the kind of security issues (see MySQL's `LOAD LOCAL`) where the server has to tell the client what local resources are needed
to continue bundling, and the client can't reason about what resources are needed on its own. Preserves original filenames and line numbers
for debugging via per-Proto source info in bytecode.

## Motivation

**Problem:** No code reuse beyond copy-paste in-viewer. Large scripts are unwieldy. LSL's `#include` loses dependency info after processing,
if you don't have the local dependencies you have no option but to edit the post-processed source code soup.

**Why static `require()`:** SL's creation and permission models don't play nice with dynamic runtime dependencies. Bundling together dependencies when
building makes it easy for us to support multiple development styles (external editor with custom build toolchain, in-viewer editor have different opinions about
how to include files and from where)

**Why it's tricky:** Server compiles, but dependencies live on client. Can't reasonably have server request files (i.e. `LOAD LOCAL` from MySQL).
Need debug info across modules. Want future tree shaking/inlining for optimization purposes.

## Path Resolution

All paths are explicit - no relative paths (`./`, `../`).

| Pattern | Resolves to |
|---------|---------------------------------------------------------------------------------------|
| `require("foo")` | `package.path`-like semantics. Includes from the top of the package, or from libs dir |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is libs dir in this instance?

If its a user defined dir, I'd personally prefer keeping that functionality strictly to the user set aliases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking it could implicitly search through the @libs alias, so that you don't necessarily have to set up a specific alias for every single package you install. In other languages terms, where @libs is equivalent to your node_modules or dist-packages or what have you. I imagine the same would work for in-world editing where you could either

a) have a "modules" subfolder with symbolic links to what you want to use for that particular folder of scripts, but this gets weird in script-inside-object cases since you can't have subfolders there
b) just load from some user-defined global "modules" folder in user inventory

whichever turns out to be less annoying for people's usecases, will need to do some investigation there. Option B seems the most obvious and frictionless, to me at the expense of things getting weird if you have two projects that both want different, incompatible versions of the same library and you want to manage your dependencies in inventory. Anything more than that and you end up effectively having to implement a package manager in the viewer for the developer experience to be acceptable.

Copy link
Contributor

@WolfGangS WolfGangS Jan 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scripters currently are used to specifying a system folder to the fs_preprocessor I think having them provide one explicitly and using @libs/.. instead of magically including something would be preferable.

Automagic choosing of a source feels like it will end up eventually conflicting with something.

For instance the viewer needs its own built in resolver for some aliases to do bundling into the package for users that don't user external tools. that would need to be disabled if the user wants to use the external vscode plugin, unless the plugin can provide a bundle where the dependencies are included in it, without aliases, so the viewer wont resolve those to anywhere but things already explicitly in the bundle.

So a save in vscode would produce this something like this.

--[[!!SLUA:BUNDLE!!]]
local a = require("thing")
print(a())
--[[!!SLUA:MODULE:thing!!]]
return function()
    return "test"
end

The viewer would do no further bundling there, as "thing" was provided, if it wasn't it should error imo, rather than behaviour being different if it is and isn't provided.

Essentially the viewer / server bundling steps only to pull in defined code with @ aliases, not "magic" that could get something not well defined by the user.

So a require("test") would error unknown module if that is not explicitly included by the user/plugin in the bundle.

| `require("@myalias/utils")` | User-configured alias → local path |
| `require("@sl/json")` | Platform libs (reserved namespace for LL-provided native Lua libs) |

**Client resolves aliases before bundling.** Bundle stores alias paths; runtime does simple string lookup when actually
calling `require()`

## Bundle Format

Text-based, valid Luau syntax, lightly inspired by MIME multipart RFC:

```lua
--[[!!SLUA:BUNDLE!!]]
-- NOTE: May have some metadata in the header too
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the meta data just any comments after the bundle comment?
Or explicit with some sort of --[[!!SLUA:COMMENT: <multiline comment text> ]] style comment?
Will they be removed for the purpose of compile and errors?

Alternatively as the bundle comment is a multi line one, anything after the first newline inside the comment could be ignored by the bundle system e.g.

--[[!!SLUA:BUNDLE!!
  <meta info about bundle>
]]

--[[!!SLUA:MODULE:@libs/lib!!
  <meta info about module>
]]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, the metadata would be part of the slua bundle directive itself.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could change the line in the rfc to this then to show it explicitly?

--[[!!SLUA:BUNDLE!!
   NOTE: May have some metadata in the header too
]]

-- MAIN is implicit (first section after header)
local foo = require("@myproject/lib/foo")
local json = require("@sl/json")
foo.bar()

--[[!!SLUA:MODULE:@myproject/lib/foo!!]]
local helpers = require("@myproject/lib/helpers")
return { bar = function() return helpers.helper() end }

--[[!!SLUA:MODULE:@myproject/lib/helpers!!]]
return { helper = function() return "hello" end }
```

**Rules:**
- `--[[!!SLUA:BUNDLE!!]]` header must be first line
- MAIN is implicit (content between header and first MODULE)
- `--[[!!SLUA:MODULE:path!!]]` marks each dependency
- `--[[!!SLUA:` in user source is rejected
- Platform libs (`@sl/...`) not included - provided by runtime
- Generally the user only sees and directly edits the `MAIN` bit of the bundle
- Users may define their own global aliases that refer to particular libs on their disk
- - For ex. you might `require("@textlib/v2")` to pull in v2 of your text rendering library
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which default file semantics will be used for the luau file if a directory is specified instead of a file directly?

At the moment it seems luau rfcs state either module.luau or init.luau I can't find a clear answer on which.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm open to either, may have to look at Lute to see if they have different resolution strategies there.


Conceptually, the bundle provides a sort of virtual filesystem for the runtime `require()` implementation.

Notably, since the bundle format includes the source code as-is, before any optimization and tree-shaking, file name
and line mappings for errors are automatically correct, without `--!@line` directives or similar.

## Editing Workflow

```mermaid
flowchart TD
subgraph Start
direction LR
OpenExisting[Open existing script]
CreateNew[Create new script]
end

DownloadArchive[Download source archive]
IsValidBundle{Valid bundle?}
MainView[MAIN-only view]
RawView[Raw view]
UserEdits[User edits]

CreateNew --> MainView
OpenExisting --> DownloadArchive
DownloadArchive --> IsValidBundle
IsValidBundle -->|Yes| MainView
IsValidBundle -->|No| RawView
MainView --> UserEdits
RawView --> UserEdits

UserEdits -->|Toggle view| CheckUnsaved
UserEdits -->|Save| StartSave

subgraph Toggle[Toggle View]
CheckUnsaved{Unsaved changes?}
TogglePrompt{Discard or cancel?}
CheckToggleDir{Raw to MAIN?}
ValidBundle{Valid bundle?}
DoToggle[Toggle view]
ShowParseError[Show parse error]

CheckUnsaved -->|No| CheckToggleDir
CheckUnsaved -->|Yes| TogglePrompt
TogglePrompt -->|Discard| CheckToggleDir
CheckToggleDir -->|No| DoToggle
CheckToggleDir -->|Yes| ValidBundle
ValidBundle -->|Yes| DoToggle
ValidBundle -->|No| ShowParseError
end

subgraph Save
StartSave[User saves]
IsRawView{Viewing raw?}
SendRaw[Send as-is]
HasLocalDep{Local dep exists?}
CheckOwnership{Same user last saved?}
BundleLocal[Bundle in local dep]
CheckMatch{Local matches bundle?}
ConfirmUntrusted{Confirm: pull local dep\ninto untrusted script?}
DepInBundle{Dep in existing bundle?}
BundleFromBundle[Bundle from existing]
ResolveError[Error: can't resolve]
SendBundle[Send bundle]
AbortSave[Abort]

StartSave --> IsRawView
IsRawView -->|Yes| SendRaw
IsRawView -->|No| HasLocalDep
HasLocalDep -->|Yes| CheckOwnership
CheckOwnership -->|Yes| BundleLocal
CheckOwnership -->|No| CheckMatch
CheckMatch -->|Yes| BundleLocal
CheckMatch -->|No| ConfirmUntrusted
ConfirmUntrusted -->|Yes| BundleLocal
ConfirmUntrusted -->|No| AbortSave
HasLocalDep -->|No| DepInBundle
DepInBundle -->|Yes| BundleFromBundle
DepInBundle -->|No| ResolveError
ResolveError --> AbortSave
BundleLocal --> SendBundle
BundleFromBundle --> SendBundle
end

TogglePrompt -->|Cancel| UserEdits
ShowParseError --> UserEdits
DoToggle --> UserEdits
AbortSave --> UserEdits

SendRaw --> Compile
SendBundle --> Compile

subgraph Server
Compile[Compile & store]
Done[Return result]
Compile --> Done
end
```

**Key points:**
- MAIN-only view: user sees entry script, client bundles on save
- Raw view: user sees/edits full archive, sent as-is
- Ownership check: security measure to prevent leaking local code into scripts others control
- - Open question as to how this should work, we'll need to do some server-side enrichment
- Fallback: if local dep missing, use version from downloaded bundle
- - This allows users to do one-off edits to scripts from the viewer even if they don't have all the
constituent parts on their drives.

## Runtime

- `require()` implemented in C (hides module table/cache from scripts)
- Module results cached after first execution
- Each module runs in sandboxed environment via `dangerouslyexecuterequiredmodule()`
- - This function will be hidden from view and not directly usable.
- Simple string lookup - no alias resolution at runtime

## Bytecode Extension

**Problem:** Standard Luau shares one `chunkname` across all functions. Bundles have multiple
source files. It's useful to be able to have proper filename and line mappings in errors.

**Solution:** Per-Proto source in bytecode. Each function stores its source filename. Loader reads per-function source for correct stack traces.

```cpp
// In lvmload.cpp
TString* protoSource = hasPerProtoSource
? strings[readVarInt(data, size, offset)]
: source; // Fallback for non-bundles
p->source = protoSource;
```

## Errors

| Error | When | Message |
|-------|------|---------|
| Dynamic require | Compile | `require() argument must be a string literal` |
| Relative path | Compile | `relative paths not supported` |
| Unknown alias | Compile | `unknown alias '@foo' in require` |
| Module not found | Compile | `cannot find module '<path>'` |
| Circular dependency | Compile | `circular dependency: <path1> -> <path2> -> <path1>` |
| Delimiter in source | Compile | `source cannot contain '--[[!!SLUA:'` |
| Depth exceeded | Compile | `require depth exceeds maximum` |
| Module not in bundle | Runtime | `module not found: <path>` |

## Limits

- Max dependency depth: 100
- Max total modules: 1000
- No circular dependencies (no different from typical Luau here!)

## Future Work

- Tree shaking (eliminate unused exports)
- Cross-module inlining (`--!pure` modules)
- Inventory-based module resolution
Copy link
Contributor

@WolfGangS WolfGangS Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this have a namespace reserved in advance?

  • @my-inv for user inventory?
  • @inv for current object inventory?

It may be prudent to declare @sl-* as reserved to give LL room to extend for more than just a direct library of scripts, without scripters hitting conflicts in the future.

For instance if inventory based requires don't receive an alias ahead of time, reserving @sl-* would allow avoiding conflicts with user made aliases by using @sl-inv to add inventory support.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're reserving @sl-*, so @sl-inv might be the way to go here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That should probably be added to the RFC then.
At the moment it only states @sl for possible LL provided libraries.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned about specifying where to pull things from instead of what to pull inside the actual code. What would @inv mean when running a test script against your code locally, or using a local bundler (as in the VSCode plugin)?

If the answer is "@inv/foo is equivalent to foo when executing locally", does that mean the only difference with @inv would be that it would never try to resolve to a local file if compiling an in-world script?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's sort of why I was thinking @sl-inv or @sl-object, local bundler like vscode, would basically ignore it, and pass it to the viewer, the viewers bundler should see it and go, Ahh, I need to go add that from the object/avatar inventory (depending on whats implemented for that), anything with @sl/ or @sl-*/ would be fairly obvious to any local tooling to leave to the sl ecosystem.

Like your bit about Platform libs (@sl/...) not included - provided by runtime.

Basically the way i picture it on save the viewers built in bundler code will try and resolve any @ aliases with its own list of rules, be that for local system folders, or avatar inventory folders or so. And the viewer have a config area on script development or so to specify other non @sl-* aliases

  • @sl-inv would look in the avatars inventory scripts folder.
  • @sl-obj would look inside the current object the script is being saved in. (would throw an error on save when saved a script in inventory with unknown alias or something)
  • @libs would check viewer config for what local system folder has been defined for libs
  • @my-http-lib same as above but for what was defined against my-http-lib