5
$\begingroup$

I've inherited maintenance duties for a domain-specific language. This language started life forty years ago as a quick way to insert the results of mathematical operations into text documents, and has grown by accretion since then.

The first thing I'd like to do is document the syntax of the language -- the real syntax, not the "handed down by word-of-mouth for forty years" syntax. Is there a method for doing this that's easier than trying to infer the syntax from 10,000 lines of poorly-commented parser-interpreter?

$\endgroup$
6
  • 3
    $\begingroup$ To be clear, the problem here is that you have the code which implements the DSL, and you want to figure out what its syntax is from the code? Not you have the syntax and you need to write a formal specification for it. $\endgroup$
    – kaya3
    Commented May 23, 2023 at 20:48
  • $\begingroup$ is there any existing code written in the language you can look at? If there is, I think that's a good way to start. Look at existing code, look at specific features, and try to trace those to code in the parser-interpreter, and just observe what that code does when it is run. Try extracting code into smaller programs, getting those to be well formed, and then observing their behaviour. I'm holding off on writing an answer post because I'd like to see what people think of the quality of this question and if it's a good fit for this site. $\endgroup$
    – starball
    Commented May 23, 2023 at 21:04
  • 2
    $\begingroup$ I’m voting to close this question because it's not about programming language design and implementation. We aren't redesigning the language here, and we already have an implementation (even if poorly written.) $\endgroup$
    – Isaiah
    Commented May 23, 2023 at 21:39
  • $\begingroup$ A question about how to represent the syntax in a formal or useful way given knowledge of the syntax would be on-topic, but decoding the syntax from existing code is neither a design nor implementation question and falls more in the realm of software engineering. $\endgroup$
    – Isaiah
    Commented May 23, 2023 at 21:40
  • 1
    $\begingroup$ @kaya3, formal specification will be the next step, hopefully followed by replacing the parser with one that doesn't crash if you look at it funny. $\endgroup$
    – Mark
    Commented May 23, 2023 at 22:33

1 Answer 1

5
$\begingroup$

I once had the same task but simpler: I needed to reverse-engineer TypeScript grammar in the form of an ANTLR grammar. You might think TypeScript has a grammar, at least informal one, but no, it's outdated for ages and new syntax constructs were only described in release notes. By the way I bet ANTLR grammar is still incomplete though I haven't checked. :)

Anyway, what made it simpler than your case is TypeScript has a huge set of tests. Thus I ran my incomplete grammar over tests and looked at the examples that failed to parse, then updated the grammar. And then another loop of the same algorithm.

I guess you don't have tests, but generating a set of tests given a grammar sounds like something a compiler fuzzer can do, though I can't provide any links. But given such set of valid examples according to your candidate grammar, you can run actual interpreter or compiler on them to see which are indeed correct and which are not. Then update the grammar accordingly. Rinse and repeat.

$\endgroup$
2
  • 1
    $\begingroup$ if you're going to fuzz for valid tests to revers-engineer a grammar, why not just fuzz for the grammar directly? (that's what I was going to propose... more as a joke though. I'm not really more efficient than just trying to read and understand the source) $\endgroup$
    – starball
    Commented May 24, 2023 at 7:30
  • $\begingroup$ I suspect writing down grammar by hand guided by parser failures in (presumably) an intelligent way is a more efficient fuzzing strategy... 😅 $\endgroup$ Commented May 24, 2023 at 7:36

Not the answer you're looking for? Browse other questions tagged .