How do languages support executing untrusted user code at runtime?

Question

Many dynamic languages such as Javascript, PHP and Python have a built-in eval function for executing code from a string. It's well known that such functions are dangerous because arbitrary things could go wrong if untrusted user input makes its way into the string that gets executed. Malicious code could modify the program state or perform other unwanted side-effects; attempt to allocate all of the available memory; overflow the stack; or loop infinitely.

However, despite their danger, programmers still use eval functions for convenience, and there are some niche uses (e.g. automated grading of programming assignments) where executing untrusted code is inherent to the task. So:

• For languages which do have a safe eval function, how is that safety ensured by the design of the language and/or the eval function's API?
• Or otherwise, what work has been done to identify how a language with a safe eval function could be designed?

To be clear, this question is not about OS-level mitigations (e.g. running untrusted code in a separate process as an unprivileged user), and not about about the implementation (e.g. how to algorithmically check whether some input code could perform a banned operation). I'm asking about how eval could be made safe by design, either in existing languages or in prior work attempting this.

Answer 1

Without a clear threat model, this is borderline too broad, but there are several attested approaches that attempt to provide some general level of safety. It's also worth noting here that most of the issues of eval are exactly those of dynamic loading of code in general, so systems that give protection there also address string evaluation. A limitation in terms of real-world examples is that eval is so deeply deprecated that languages that care simply don't have it — but they often do have dynamic loading.

Object Capabilities

The object-capability model, in short, is that holding a reference to an object indicates that you have permission to access all of its operations, and so operations that are not universally available can be encapsulated within an object. That object can be guarded carefully, and when access to some of the protected actions should be given, a wrapper object exposing only those operations can be created and given to other code.

A language based around this approach will not expose any access to the outside world other than through an initial global capability, which can be refined to restrict the access of any code, not only evaled code, including both dynamically-loaded code and code written within the program itself. Whenever code is executed it must be provided the capability or capabilities it will be permitted to use — possibly as ordinary arguments, or possibly through a capability-specific mechanism of the language — and it's certain that that code can't do anything beyond those, and can't trick other functions into doing those tasks because it would need to provide that function a valid capability to allow that.

Notable languages following this model include Emerald, Joule, E, Newspeak, and Wyvern. Google's Caja system for JavaScript also used it. It's an active area of research. It's possible to build implementations of this system on top of most OO languages, but if the language itself provides escape hatches (e.g. language keywords for file access, loading other code, and so on) then it has limited effect.

Despite the name, this doesn't strictly need to be in an object-oriented language: an opaque token that is passed around and tracked by the runtime can provide exactly the same facility, though defining restricted subcapabilities just by creating objects using the ordinary features of the language is a major convenience. The broader area of capability-based security encompasses all approaches here.

Privilege dropping

Another quite-related model is to allow the program to drop certain abilities at will, without providing the ability to transfer them around. This is broadly aligned with what POSIX calls "capabilities" (which are different to the object-capability model and general capability-based security), but at the linguistic level. The program can choose to drop the ability to open files, for example, either permanently or for the dynamic duration of a particular function or block. If code is evaled during that time, it won't be able to do that either.

This is in essence what the Java Security Manager was giving, and was part of the original proposition for Java. It has some significant limitations, particularly that it is a negative instead of affirmative system: it's not resilient against new privileges being added in the broader system.

Environment provision

Many languages' eval allows providing a scope in which to execute the code. This scope can contain variables to be read or modified, and often definitions of functions as well. Python is one such language: passing in a dictionary of globals and/or locals restricts access to the true values, and can shim standard functions. This is a core pattern in Lisp, and Tcl is another that provides fine-grained ability to control the evaluation environment.

Purity

A pure functional language inherently exposes no unfettered access to the outside world, and so evaluating untrusted code is "safe" from those impacts. However, infinite recursion or repetition may still be possible. There are pure functional languages that provide this (even Haskell can, via dynamic loading), though they are not very common.

Totality

To avoid the runaway-repetition scenario, a language can enforce that all valid code be total and provably terminate. Agda is such a language. These languages could provide eval safely, although I can't find any that do, probably because it's so clearly a bad idea.

Static effect systems

An effect system, broadly, identifies the computational effects of code: what side effects can occur, and where. These propagate, such that code that has effect X (such as writing to a file) contaminates any code that calls it with the same effect, unless somehow suppressed by a language construct. A familiar (limited) example of an effect is Java's checked exceptions, but other common effects modelled are file and network access (sometimes including specific resources), memory mutation, and control flow.

A language exposing a two-step eval process can leverage this: parse and process the code into an unexecuted value, which is annotated with its effects. Client code can then examine the value and determine whether to run it or not based on the effects determined. Execution can even be prohibited at the language level when the "execute" operation has correct effect annotations.

I'm not aware of a language that does this specifically for eval, but it is part of the dynamic loading system of Unison, and arguably of Java in the case of exceptions.

Domain-specific languages

Rather than exposing evaluation of the same language as the host, a more limited language can be exposed for the cases where evaluating user code is required. These sublanguages can then have requirements of totality, purity, and so on that are not imposed on the host language. This is in effect what the Common Expression Language is to Go, although it is formally an external library. Python provides ast.literal_eval for interpretation of literals only (but not expressions on them).

A restricted eval reusing only expression-oriented parts of the language's parser and running on provided data is a fairly cheap extension to an interpreter. Strictly speaking, this is also what many languages' regular-expression systems are, although they are not usually computationally useful in that way.

Operating-system-level isolation

You wanted to rule this out of scope, but overall "safety" inherently can't be guaranteed purely at the level of the language runtime. Process isolation is one strong boundary against escapes through implementation bugs, but OS-level capability systems with linguistic support (as in Midori) also help. How much this is necessary or helpful depends on the threat model involved.

Sandboxing and timeouts

The original proposition of Java was that it would be safe to run untrusted code within other systems due to sandboxing. It didn't work out as well as intended, but this is a valid approach in combination with other methods. The most straightforward approach to preventing nontermination is simply to impose a time limit, either at the system level or linguistically. These are the obvious approaches that aren't worth more detail, but they are a realistic part of any practical system.

Nothing

In practice, this is overwhelmingly the most common option. Languages that add eval itself today mostly don't care about these issues: they treat it as the programmer's responsibility to be safe and tell them they shouldn't use it.

Expose alternatives for common cases

Richards et al. found in 2011 that the principal uses of eval in JavaScript code were for dynamically accessing language facilities like field access or assignment, or for JSON parsing. Many of these were either mistakes or design flaws in the code, such as faking an array with a sequence of variables eval("subPointArr_" + i), or evaluating a field access instead of using [] indexing.

A number of uses were for tasks that the standard library or ecosystem now provide inbuilt and safe methods for, like JSON parsing and deferred loading of scripts. Many uses were function calls with dynamic argument lists, which were possible at the time without eval, but awkwardly (via Function.prototype.apply), and are now exposed as a language feature using spread syntax.

A very large proportion of the uses of eval found are thus no longer necessary (or weren't ever). The safest eval is the one that doesn't happen, so it may be that the best way to make eval safe is to eliminate the cases that require it, and then remove the functionality itself. This is probably the best language-design pathway if aspiring for safety.

_{Richards, G., Hammer, C., Burg, B., Vitek, J. (2011). The Eval That Men Do. In: Mezini, M. (eds) ECOOP 2011 – Object-Oriented Programming. ECOOP 2011. Lecture Notes in Computer Science, vol 6813. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22655-7_4}

Answer 2

Safe TCL

TCL supports this as safe TCL

In TCL all control structures are implemented as commands and any command can be replaced.

In TCL you can create a new interpreter from within TCL itself which only supports the safe subset.

Here is a paper from 1998 that describes this in some detail.

Other Approaches

The basic approach there is to create a restricted more secure subset of your base language.

You could also consider:

• A capabilities model

• Limiting usage of resources (memory, threads, files etc.)

• Runtime_verification

• Provable_security

These improve on devising a safe subset by adding compile and/or run time checks that the untrusted code is safe for some measure of safe.

Consider other Sandboxing Approaches

There is an interesting discussion for the case of python here:

https://stackoverflow.com/questions/3688708/python-safe-sandbox

Maybe eval is too general?

Eval is often much more general that you require. For example in the case of attacks like SQL injection maybe you don't need to be able to execute an arbitrary query maybe its enough to specify a table and key.

An example of similar kind of injection attack can occur if you delegate to something via text like

 doSomething --parseString $SomeStringInFormatX

In this case you could possibly attack via something like:

 "parseThis --acommand line option ;a new command altogether"

Using -- to start another command line option or ; to start a new command if doSomething is evaluated directly. Whereas if the code to be parsed is in a file the attack surface is limited to the capabilities allowed when parsing a file of type X.

So you could instead say:

 doSomething --parseFile $SomeFileInFormatX

Or better skip the text altogether and have a function:

 doSomething(SomeFileNameInFormatX: path)

---

I should add that many of these approaches have risks. In TCL, for example, you can add commands to the inner interpreter and make it unsafe. Also I don't know know if there is any proof that the "safe" subset cannot be circumvented.

The only surefire way to secure untrusted code would be running proofs on untrusted code and that is a hard problem.

Look for references like for example - https://www.cylab.cmu.edu/news/2022/10/11-provably-safe-sandboxing.html

---

Also you mention making eval safe by design and specifically not considering OS-level mitigations but you simply cannot ignore those.

For example, if you allow memory allocation than you will have to provide a way limit it as a function of your runtime or you allow a attack that will exhaust your memory. Sometimes the OS can help you there. For example, with posix and linux you can set resource limits. If the OS doesn't go far enough you will have to add similar things to your runtime. The more you do that the more your runtime will begin to ressemble an OS in some aspects. It is a perfectly valid approach to say I don't want to have to worry too much about that and instead use external sandboxing solutions be they OS sandboxes, containers (docker etc), unikernals or virtualisation. As an added bonus you can potentially 'trust' systems that aren't part of your language at all. Using docker to implement 'eval' might be a bit of a sledgehammer but that sledgehamemr is a useful tool in its own right.