3
$\begingroup$

I'm mostly looking to poke holes in this design, as well as suggestions on next steps.

For the language I'm creating, I think I might have additional keywords and statements available just for test files.

What are the issues with this?

test suite  // not sure what exactly to name this, but it matches the file
//  vv these are modifiers that apply to the test suite. The language has similar modifiers for classes, fields, and functions.
    covers(Driver)  // this is the class under test, and in an IDE could be navigable
    tag(unit)   // could be an arbitrary/configuration-defined list of tags (system, integration, long-running, etc)
{
    beforeEach  // runs before each test variant
    {
        Driver driver <- Driver.new()
    }
    
    // declares a test with the given name (which is a free form String)
    test Should have full name
    //  vv this is a modifier for the test
        covers(new(String, String)) // the field or function under test
    {
        variant Valid   // a name for the variant of this test
        {
            arrange
            {
                String firstName <- "Bugs"
                String lastName <- "Bunny"
            }
            act
            {
                Driver testDriver <- Driver.new(firstName, lastName)
            }
            assert
            {
                // this example just reuses something like FluentAssertions for now
                testDriver.firstName.should().be(firstName)
                testDriver.lastname.should().be(lastName)
            }
        }

        variant Empty names
        {
            arrange
            {
                String firstName <- ""
                String lastName <- ""
            }
            act
            {
                Driver.new(firstName, lastName)
            }
            assert (RequirementFailedError error)   // expecting an error to be thrown in the act block
            {
                error.message.should().be("Driver must have a first and last name.")
            }
        }
    }

    test Should be buckled
        covers(drive())
    {
        beforeEach  // a beforeEach or beforeAll could be inside the test suite or the test
        {
            Driver testDriver <- Driver.new()
        }

        variant Buckled
        {
            // if a block is only one line, it could just follow the keyword.
            arrange testDriver.buckle()
            act testDriver.drive()
            assert testDriver.isDriving
        }

        variant Not buckled
        {
            act testDriver.drive()
            assert (RequirementFailedError error)
            {
                error.message.should().be("Driver must be buckled first.")
            }
        }

        variant Unbuckled
        {
            arrange
            {
                testDriver.buckle()
                testDriver.unbuckle()
            }
            act testDriver.drive()
            assert (RequirementFailedError error)
            {
                error.message.should().be("Driver must be buckled first.")
            }
        }
    }

    test Should be at least 16
        covers(age)
    {
        variant Valid age
        {
            // this is using the driver declared in the test suite beforeEach.
            arrange Natural age <- 16
            act driver.age <- age
            assert driver.age.should().be(age)
        }

        variant Underage
        {
            act driver.age <- 15
            // if it throws a different Error, then the test fails (along with the details of the Exception).
            assert (RequirementFailedError error)
            {
                // Here, the details of the Error can be verified.
                error.message.should().be("Requirement (age >= 16) was not met.")
            }
        }
    }
}

(If you're finding this a bit hard to read with the comments and lack of syntax highlighting, here you go.)

My thought is that there are some benefits to having the framework know what class/function/field is under test, as well as knowing what's in the arrange block, the act block, and the assert block.

I know that I'll need a way to do parameterized tests.

Later, I can also optionally build a mocking/stubbing framework into the language too.

There's also an opportunity to add some functionality to the assert block. (So that an assertion library is not needed.)

$\endgroup$
4
  • $\begingroup$ I can't think of a situation in which this would be advantageous. What extra information does the language have that allows it to test better over an external test framework? (Assuming my interpretation of the test mechanism as being built into the language is correct, and you're not just adding dummy keywords like Kotlin did for async.) $\endgroup$
    – Ginger
    Oct 5, 2023 at 19:52
  • $\begingroup$ "My thought is that there are some benefits to having the framework know what class/function/field is under test, as well as knowing what's in the arrange block, the act block, and the assert block." ─ can you tell us what those benefits are? $\endgroup$
    – kaya3
    Oct 5, 2023 at 20:02
  • $\begingroup$ @kaya3 When executing the tests, if an exception happens in the arrange block or the assert block, it might not be function under test that actually erred, and the failure message could show this. I have mixed thoughts on this, but the assert block could reuse the core framework, rather than needing an external library that restates the core framework. (For example, assert items.contains(item) rather than items.should().contain(item).) The arrange/act/assert blocks built in means that the format is strongly encouraged by the language. It could error if some of those blocks are not present $\endgroup$
    – Xtros
    Oct 5, 2023 at 21:12
  • $\begingroup$ I kind of like this for small tests of basic features. I've been using it in a project I've been working on. I like that it allows you to package together the tests and the definitions that the tests use. That way, you only need to look in one place to see both. Am I accidentally using a nonexistent function? I can look in one file to determine this. I want to see if I've tested every function? I can look at one file. I want to add a new function and tests for it? I change one file. It's seems easier to keep the two aspects up-to-date with each other. In this case, I frequently change both. $\endgroup$ Oct 5, 2023 at 21:26

0

Browse other questions tagged .