Feature idea: serialize from required CJS modules

[harrysolovay]

Hi Blake,

I have a feature idea: serializing cross-file. Even with the references option enabled, requires/imports evaluate to their references (not the fully-serialized data).

Worse comes to worse, one could just bundle their code and then execute this lib's stringify fn. Although, ideally, this build step could go away.

Here's a simple example of where this might come in handy.

Identifying the difference between node & browser environments seems to be a reason not to implement this... although I'm sure there're good workarounds.

Anyways, please let me know your thoughts on this. Would love to discuss implementations & work on a PR :)

[blakeembrey]

Based on the current API, this isn't possible (as far as I'm reading the request). It isn't smart enough to be a complete serializer and parse references, or have access to the original source code. For this feature to work, I think you'd need to know which file a comes from, then parse it using a JavaScript parser for variable references out of scope, then use that to discover their expressions. Does that sound like the request here? I'd be hesitant to blow up scope of this package like that, since the basic serializer is assuming (if you wish for the code to run) that all variables are defined or within scope once serialized.

[harrysolovay]

I completely agree: parsing other files in search of bindings would be a bad design decision.

I'm trying to think of another approach... I'm not sure this is particularly better, but how would you feel about an approach similar to what you do in ts-node: (if a special flag is enabled) you can create a virtual module, into which you place the current file's contents. As other modules get required, a require extension injects their contents along side the current file's contents in the virtual module. Finally, the virtual module can be used as a source of truth for javascript-stringify. Still feels like a grave workaround...

Do you see any other way that this feature might come to life?

[blakeembrey]

It depends on what exactly you're trying to achieve. The easiest way would be to accept a "dependencies" dictionary that is also stringified in scope. It would be less advanced, but also a lot simpler. E.g. stringify(x, { dependencies: { a } }) where a may be referenced from within x.

[harrysolovay]

I like that solution! But it forces you to manually capture and place the serialized data... could the next step be to add an option that allows users to specify how to retrieve dependencies? E.g.:

stringify(x, {
  getDependency: ref => {
    // retrieve the serialized reference here... perhaps from another module
    return `{b: {c: "d"}}` // or whatever
  },
})

... this would make javascript-stringify pluggable! Given this ability, I could work on a plugin that does the cross-module serialization

[harrysolovay]

What'm I thinking! I can just use a custom replacer. Foolish of me to suggest the above ^. I'll run an experiment & get back to ya!

[harrysolovay]

Finally ran the experiment and it didn't go great. I think the answer is utilizing bundlers beforehand. NCC is pretty good for this use case. Still, would be nice if there was a more first-class way.

How would you feel about something that taps into V8 Stacktracing? Here's an example of how Pulumi solves this problem.

Serializing cross-module (and being able to specify which modules to exclude) feels like an incredibly useful feature.

[blakeembrey]

Unfortunately not every environment is V8 so that wouldn't be possible. Would you mind sharing a repro that I can take a look at to understand why custom replacers failed?

[harrysolovay]

Makes sense. I tried logging the FunctionParser instance within your functionToString, but could not find any references to the nested function calls. Nor was I able to access the stack, which you define at the top level of stringify. I found that I'm able to .toString functions from other modules. I'm having difficulty figuring out where the serialization stops (certainly has to do with my lack of understanding this approach).

As for a good mechanism for excluding imports (and this is probably out of scope for this project). One could import and tack on a symbol to the module they wish to exclude, and then inject it back into the require cache (assuming they're in Node). Then, within the replacer, they can return undefined when the given value/module with the injected symbol is encountered. In my use case, such a symbol would indicate that the corresponding module should be "required" (its require statement concatenated with the serialized result).

What're your thoughts? Also, if this simply doesn't relate to the core functionality, please feel free to ignore & close this issue :)

Thank you Blake!