add file content #577

BrentBlanckaert · 2025-01-11T19:19:17Z

No description provided.

…make stdin more consistent

… consitent

BrentBlanckaert · 2025-02-18T18:45:26Z

Documentation for usage of files

This documentation will discuss all the changes made regarding
the IO of a test.

Files

Files are used to describe files that can be added as input for a test and will provide the student with this file in Dodona. This is done in the following way:

files:
  - name: "animal.txt"
    url: "media/workdir/animal.txt"

files is not to be confused with file which will specify the contents and location of a file the code of a student should generate. This can be done in the following way:

file:
  content: "animal.txt" # is the content that the file should have
  location: "media/workdir/animal.txt" # is the location of the file
  oracle: ...

There are several issues with this:

The usage of the names files and file is very confusing
In content you had to use a file and can't just specify the content
You're able to have multiple input files but can only have one output file.
Need more consistency in the naming and formatting

The name files was changed to input_files and file was changed to output_files. The name url in files and location in file were also changed to path. You can now also specify multiple files for the output files. An example is the following:

input_files:
  - name: "animal.txt"
    path: "media/workdir/animal.txt"
  - name: "human.txt"
    path: "media/workdir/human.txt"
output_files:
  data: 
    - content: "lion" 
      path: "media/workdir/animal.txt" 
    - content: "tim"
      path: "media/workdir/human.txt"
  oracle: ....
output_files:
  - content: "animal" 
    path: "media/workdir/animal.txt" 
  - content: "humant"
    path: "media/workdir/human.txt"

You can still only specify paths in the content section of output files.
We can distinguish between actual content and the path the a file that contains it by using !path.
An example is the following:

output_files:
  data: 
    - content: !path "animal.txt" 
      path: "media/workdir/animal.txt" 
    - content: "Humans can make music and a warm meal"
      path: "media/workdir/human.txt"
  oracle: ....

So content will now expect the actual content by default and not a path to load it from.

For the feedback, it's all still a bit fuzzy because right now all the content is dumped after eachother. Potential solution:

Usage of tabs in the solutions
Only show the names and show content when clicking on them.

Most of that will probably need to happen on Dodona itself.

Stdin, Stdout and Stderr

How things currently work, you have to specify the full contents of the stdin, stdout and stderr channels. This can get ugly, when that's a lot of text. This is why the usage of files is also very benificial here.

Example for Stdin:

stdin: !path "media/workdir/animal.txt"

The usage of !path is also present here. This is consistent with what was discussed above.

Under the hood, Stderr and Stdout are both just textual output channels. So they both have the exact same functionality.
If they are a dictionary, they used to expect the key data, but now you can also use content which is more consistent with the rest. Just like before you can also use !path to specify that you want to use a file instead of directly specifying the content.

Examples for Stdout:

stdout: !path "media/workdir/animal.txt"

stdout: 
  content: !path "media/workdir/animal.txt"
  config: ...
  oracle: ...

jorg-vr · 2025-04-25T13:43:47Z

Thanks for all the detailed examples. I'll try to follow the same structure in my feedback.

Input files

I would remove the parameter url. I know we require it right now, but we are trying to improve this for the better, and this causes needless duplication. I'll go into more depth on why in the examples.

First example

This example makes no sense. All benefits of being able to specify content inline are lost if you also specify a url. Because now we would still need to create a file with that content in the media dir. (And it makes no sense for that content to be different from the content specified inline)

First example feedback

I would expect the data.input_files to contain the content instead of the url.

Second example

Why are we providing urls to the random internet here? This again makes no sense. You're exercise will be using a local copy of that file in the git repo. But students would be linked to a random website, which content might change, breaking the exercise. These should not be a use-case we want to support.

some more feedback

If fish.txt contains the content, you could simply only provide path.
We can then either expect a file with the same path to also exist within the media dir, or I could update Dodona, to make the files from the workdir also available by link if they are specified as input files.
If we don't want to do to many changes at once, we can currently keep urls here. (But it should be media dir urls)

Another option would be to be more consistent with output_file.
Always require both path and content.
The path is the location in the workdir where the student can expect the file to exist.
The content is the content of that file.
If the content becomes very long, you can specify a content: !path with a path within the exercise directory. If this path is in the media dir, the content will be also be downloadable on dodona, otherwise it won't be.

Output files

I am not sure I am the biggest fan of the fact that path and content: !path, are both called a path. But I have no immediate better suggestions.

First example

I assume channel was already used in this way for stdin and stdout?
I think this is fine.

Returning the content inline in all cases is fine for now. We can resolve exceptionally large files in a future iteration. (But Dodona isn't very suited to compare long files anyway)

Stdin

I like that stdin matches input files. So, feedback applied there should also be usefull here

first example

ok

second example

I don't see much need for this usecase (as it is supported by first example)
But i am not completely against

I do not understand why the output would be any different from the output of the first example?

Third example

The usecase here seems, very long stdin, better specified in a file.

If we didn't require a name, we could simply specify stdin: !path path/to/file
And then we could generate $ submission < <a class=\"file-link\" target=\"_blank\">stdin</a>
This would be my preferred scenario.

If we insist on naming the file we can do

       - stdin: 
          path: "hello.txt"
         content: !path "hello.txt"

I am confused by your example, as url contains workdir, while I would expect the url to be for generating links on dodona (and thus be to the media dir). But see my suggestions on input_files for url.

Fourth example

See input files, having both content and url, makes no sense.

Fifth example

no comment

sixth example

no comment

Seventh example

How is the description adjusted to contain the link? A simple text replace?
I would expect the description to remain unchanged if one was explicitly provided by the user.

Stdout & Stderr

First example

ok

Second example

Again no need, but no harm done

Third example

I would prefer simply stdout: !path path/to/file again for consistency.

The example feedback is really confusing.
Expected output inlined is again fine for me.
But what is {"data": {"input_files": [{"path": "files_tests/hello_out.txt", "url": "media/hello_out.txt"}]}, "command": "close-context"} supposed to do?
Where does input_files come from. Why do we have a url when the actual content is inlined anyway?

Fourth example

I don't get at all how arguments should have any impact on how we represent stdout?

Fifth example

Having everything specified at once is very confusing. What will TESTed use?
There is really no point in supporting this. It might be even better to crash on confusing user input, then to do something they might not expect without any warning.

BrentBlanckaert · 2025-04-25T14:56:08Z

Thanks for your feedback.

Input files

First example

It is a valid point to scrap url when content is specified.

Second example

I just read this part of the existing documentation again.
I thought this was in fact supported, but the same example with a relative path still holds.

I'm not against the removal of url, but I would like to discuss this with @pdawyndt again before I make any changes.

Output files

I really wouldn't change anything here.
The specified channel used be just File. For stdout and stderr you would just specify stdout or stderr.

Stdin

Third example

This was a mistake. I edited the docs.
If the same changes happend to input_files, I could apply this. To me, using stdin: !path path/to/file would be the exact same as using path.

Seventh example

This is done using a regex and text replacement.
This is the exact same way as done in expressions and statements.
This is also nothing new. TESTed already modified the description before I did any changes.

Stdout & Stderr

Third example

They just aren't used in this case.

Fourth example

I'm simply following the way it's done with stdin. If stdinis specified with a file, the description will be shown using input-redirection. So if stdoutorstderr` are also specified with a file, we also add output-redirection.
The exact same thing applies when using arguments.

Fifth example

I don't understand what you mean? This is just a simple example where output-redirection for stdout and stderr are combined in the description. I'm just using data instead content, just to show that this is still possible.

jorg-vr · 2025-04-25T16:22:04Z

Stdout & Stderr
Third example

They just aren't used in this case.

Then why is there data with input_files in the output. These shouldn't be there,

Fourth example

I'm simply following the way it's done with stdin. If stdinis specified with a file, the description will be shown using input-redirection. So if stdoutorstderr` are also specified with a file, we also add output-redirection.
The exact same thing applies when using arguments.

Okay I missed that it was an output pipe and not an input pipe.
Not sure if I am a fan of this.
The fact that the expected output was specified as a file is internal logic from tested, not something we should confront the user with.

For stdin, an argument for hiding large inputs can be made. But for stdout we'll always want to visualize the comparison of expected and generated. And I don't see the need to ever show this as a file to the user.

Fifth example

I don't understand what you mean? This is just a simple example where output-redirection for stdout and stderr are combined in the description. I'm just using data instead content, just to show that this is still possible.

You have specified:

path which is a path to a file in the exercise repo with the expected content of stdout
content which is the expected content of stdout
url which is a path to a file in the media dir, which can be used to make an easy link to download the expected content of stdout

So we have now three ways to specify the expected content of stdout.
They all have their own specific meaning. But I really don't see a good usecase when you would specify all three at once. As I would expect all to essentially contain the same actual content.
And it will become very confusing when they don't contain the same content. If the content at path is different from the content in content, which will be used by TESTed? And if the content of url is also different, will the file that the student downloads then be different as well?

This creates some complexity that is not needed nor desired

BrentBlanckaert · 2025-04-25T17:05:16Z

Okay I missed that it was an output pipe and not an input pipe.
Not sure if I am a fan of this.
The fact that the expected output was specified as a file is internal logic from tested, not something we should confront the user with.

For stdin, an argument for hiding large inputs can be made. But for stdout we'll always want to visualize the comparison of expected and generated. And I don't see the need to ever show this as a file to the user.

This was an extra request from @pdawyndt. Since input redirection can already be present in the description, it made sense to consider adding output redirection as well. If we eventually support using files for both expected and generated output, then this addition could become even more useful.

You have specified:

path which is a path to a file in the exercise repo with the expected content of stdout

content which is the expected content of stdout

url which is a path to a file in the media dir, which can be used to make an easy link to download the expected content of stdout

So we have now three ways to specify the expected content of stdout.
They all have their own specific meaning. But I really don't see a good usecase when you would specify all three at once. As I would expect all to essentially contain the same actual content.
And it will become very confusing when they don't contain the same content. If the content at path is different from the content in content, which will be used by TESTed? And if the content of url is also different, will the file that the student downloads then be different as well?

This is just the exact same as input_files and stdin. When content is provided, no actual file is needed. In this case path will simply serve as a name that could be placed in the description. I do agree that url isn't actually needed in this case.

BrentBlanckaert · 2025-04-26T14:28:47Z

Documentation for new extensions (updated version from feedback)

This documentation will discuss all the changes made regarding the IO of a test.

Input files

Input files (currently files in TESTed) are a list of files defined in the DSL. Each file can be represented using three fields: path (required), url, and content. I will show a few examples to show what these do:

First example

tabs:
- tab: counting
  contexts:
  - testcases:
    - expression: count_words('fish.txt', 'sharks')
      return: 1
      input_files:
        - path: "fish.txt"
          content: "There are sharks in the water!"
        - path: "mammal.txt"
          content: "There are tigers in the water!"

A path must ALWAYS be provided. This is the string that will be shown to the student in the above expression. The path field (originally called name) contains relative path names and is relative to the workdir directory. In the case of expression, statement, or descriptions, an exact match will be checked. In the expression, that string becomes a hyperlink pointing to the specified url or in this case to the provided content.

The presence of the content field also indicates that the file does not yet exist. In this case, a file will still need to be created in the working directory.

The feedback for this example is the following:

First example feedback

{"command": "start-judgement"}
{"title": "counting", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "count_words(&#x27;<a class=\"file-link\" target=\"_blank\">fish.txt</a>&#x27;, &#x27;sharks&#x27;)", "format": "html"}, "command": "start-testcase"}
{"expected": "1", "channel": "return", "command": "start-test"}
{"generated": "1", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"message": {"description": "<div class='contains-file''><p>File: <a class=\"file-link\" target=\"_blank\"><span class=\"code\">mammal.txt</span></a></p></div>", "format": "html"}, "command": "append-message"}
{"data": {"statements": "count_words('fish.txt', 'sharks')", "input_files": [{"path": "fish.txt", "content": "There are sharks in the water!"}]}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

The seen input files are provided in the data and the start of the html to make a hyperlink is provided in the description. Dodona will use to create their own hyperlink or pop-up.

Second example

tabs:
- tab: counting
  contexts:
  - testcases:
    - expression: count_words('fish.txt', 'sharks')
      return: 1
      input_files:
        - path: "fish.txt"
          url: "media/workdir/fish.txt"

In this case, the file is expected to already be present in the working directory. The feedback would be the following:

Second example feedback

{"command": "start-judgement"}
{"title": "counting", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "count_words(&#x27;<a class=\"file-link\" target=\"_blank\">fish.txt</a>&#x27;, &#x27;sharks&#x27;)", "format": "html"}, "command": "start-testcase"}
{"expected": "1", "channel": "return", "command": "start-test"}
{"generated": "1", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {"statements": "count_words('fish.txt', 'sharks')", "input_files": [{"path": "fish.txt", "url": "media/workdir/fish.txt"}]}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

The main difference is that the url is present in data.input_files.

Output files

In TESTed, you were only able to provide a single output file using file. This has been extended to support multiple.
An output_file contains both a path and a content field, both of which are required.

The path field specifies the path to the file that the student should have generated (in the working directory).
The content field contains either the expected contents of the generated file or a path to a real file containing the expected contents (located in the evaluation folder).

First example

tabs:
- tab: output_file
  contexts:
  - testcases:
    - expression: genereer('origineel_tekst.txt', 'text', 3)
      return: true
      output_files:
        - content: !path "files_tests/tekst1.txt"
          path: "text1.txt"
        - content: !path "files_tests/tekst2.txt"
          path: "text2.txt"
        - content: "Created using write mode.\n3\n"
          path: "text3.txt"
      input_files:
        - path: "origineel_tekst.txt"
          url: "media/workdir/origineel_tekst.txt"

The feedback provided for this test would look something like the following:

{"command": "start-judgement"}
{"title": "output_file", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "genereer(&#x27;<a class=\"file-link\" target=\"_blank\">origineel_tekst.txt</a>&#x27;, &#x27;text&#x27;, 3)", "format": "html"}, "command": "start-testcase"}
{"expected": "Created using write mode.\n1\n", "channel": "File: text1.txt", "command": "start-test"}
{"generated": "Created using write mode.\n1\n", "status": {"enum": "correct"}, "command": "close-test"}
{"expected": "Created using write mode.\n2\n", "channel": "File: text2.txt", "command": "start-test"}
{"generated": "Created using write mode.\n2\n", "status": {"enum": "correct"}, "command": "close-test"}
{"expected": "Created using write mode.\n3\n", "channel": "File: text3.txt", "command": "start-test"}
{"generated": "Created using write mode.\n3\n", "status": {"enum": "correct"}, "command": "close-test"}
{"expected": "True", "channel": "return", "command": "start-test"}
{"generated": "True", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {"statements": "genereer('origineel_tekst.txt', 'text', 3)", "input_files": [{"path": "origineel_tekst.txt", "url": "media/workdir/origineel_tekst.txt"}]}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

For each output file, the feedback includes a pair showing both the expected and the generated content. The channel field in the expected output will be labeled as File: <path of output file>. The full contents of both files are included directly in the feedback, as there is currently no support for returning files themselves instead of their contents.

Stdin

In TESTed, stdin could only be a string or any other basic type. I also expanded the capabilities for that.
If stdin is a string, it is equivalent to using the field content. Alternatively, a path (relative to the working directory) can be provided along with a corresponding url for Dodona (relative to the evaluation directory). As soon as a path is present, that will be shown in the feedback instead.

First example

tabs:
- tab: stdin
  contexts:
  - testcases:
      - stdin: "hello"
        stdout: "hello world!\n"

In this setup, stdin cannot be an object, as this would cause issues in the validator. It could be mistaken for one of the next examples, leading to incorrect interpretation or validation errors. This does not mean it's no longer supported. You would just have to use the next example.

Second example

tabs:
- tab: stdin
  contexts:
  - testcases:
      - stdin:
          content: "hello"
        stdout: "hello world!\n"

This one is equivalent with the first example. The feedback for those would look something like this:

{"command": "start-judgement"}
{"title": "stdin", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "hello\n", "format": "console"}, "command": "start-testcase"}
{"expected": "hello world!\n", "channel": "stdout", "command": "start-test"}
{"generated": "hello world!\n", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

This is the exact same as it is now, but stdin is no longer present in data. This is because we didn't see any more use for it.

Third example

tabs:
- tab: stdin
  contexts:
  - testcases:
      - stdin: 
          path: "hello.txt"
          url: "media/workdir/hello.txt"
        stdout: "hello world!\n"

In this case, the content must be read from a file. The output looks like this:

Third example feedback

{"command": "start-judgement"}
{"title": "stdin", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "$ submission &lt; <a class=\"file-link\" target=\"_blank\">hello.txt</a>", "format": "html"}, "command": "start-testcase"}
{"expected": "hello world!\n", "channel": "stdout", "command": "start-test"}
{"generated": "hello world!\n", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {"input_files": [{"path": "hello.txt", "url": "media/workdir/hello.txt"}]}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

The provided data file is also given in data. The description is also modified when using files.

Fourth example

tabs:
- tab: stdin
  contexts:
  - testcases:
      - stdin:
          path: "hello.txt"
          content: "hello"
        stdout: "hello world!\n"

In this case, the file doesn't need to physically exist in the working directory. The feedback will be the following:

Fourth example feedback

{"command": "start-judgement"}
{"title": "stdin", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "$ submission &lt; <a class=\"file-link\" target=\"_blank\">hello.txt</a>", "format": "html"}, "command": "start-testcase"}
{"expected": "hello world!\n", "channel": "stdout", "command": "start-test"}
{"generated": "hello world!\n", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {"input_files": [{"path": "hello.txt", "content": "hello\n"}]}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

Now the content will be provided in data instead of a url.

Fifth example

tabs:
- tab: stdin
  contexts:
  - testcases:
      - stdin:
          path: "hello.txt"
          content: "hello"
        arguments: ["world"]
        stdout: "hello world!\n"

Because an argument was provided, the desciption will look a bit different:

Fifth example feedback

{"command": "start-judgement"}
{"title": "stdin", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "$ submission world &lt; <a class=\"file-link\" target=\"_blank\">hello.txt</a>", "format": "html"}, "command": "start-testcase"}
{"expected": "hello world!\n", "channel": "stdout", "command": "start-test"}
{"generated": "hello world!\n", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {"input_files": [{"path": "hello.txt", "content": "hello\n"}]}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

Sixth example

tabs:
- tab: stdin
  contexts:
  - testcases:
      - stdin:
          content: "hello"
        arguments: ["world"]
        stdout: "hello world!\n"

Sixth example feedback

{"command": "start-judgement"}
{"title": "stdin", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "$ submission world <<< hello", "format": "console"}, "command": "start-testcase"}
{"expected": "hello world!\n", "channel": "stdout", "command": "start-test"}
{"generated": "hello world!\n", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

TESTed uses here-files in this case, but for single-line content, you can use a special shorthand syntax, as shown in the description.

Seventh example

tabs:
- tab: stdin
  contexts:
  - testcases:
      - stdin:
          path: "hello.txt"
          content: "hello"
        arguments: ["world"]
        description: "stdin_test world < hello.txt"
        stdout: "hello world!\n"

Here the description will actually use the file provided in stdin to generate the start of a hyperlink:

Seventh example feedback:

{"command": "start-judgement"}
{"title": "stdin", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "stdin_test world &lt; <a class=\"file-link\" target=\"_blank\">hello.txt</a>", "format": "html"}, "command": "start-testcase"}
{"expected": "hello world!\n", "channel": "stdout", "command": "start-test"}
{"generated": "hello world!\n", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {"input_files": [{"path": "hello.txt", "content": "hello\n"}]}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

Stdout & Stderr

In TESTed, stdout and stderr already followed the exact same syntax. So it makes sense that that is still the case.
A few examples using stdout:

First example

tabs:
- tab: stdout
  contexts:
  - testcases:
      - stdin: "hello"
        stdout: "hello world!\n"

This was already possible in TESTed.

Second example

- tab: stdout
  contexts:
  - testcases:
      - stdin: "hello"
        stdout: 
          content: "hello world!\n"

In TESTed, the key data would be used. This is still possible, but content is a better name and more consistent with the new changes.

First and second example feedback

{"command": "start-judgement"}
{"title": "stdout", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "hello\n", "format": "console"}, "command": "start-testcase"}
{"expected": "hello world!\n", "channel": "stdout", "command": "start-test"}
{"generated": "hello world!\n", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

This is still the same as it is in TESTed.

Third example

tabs:
- tab: stdout
  contexts:
  - testcases:
      - stdin: "hello"
        stdout: 
           path: "files_tests/hello_out.txt"
           url: "media/hello_out.txt"

Just like stdin, you can provide a url with a path.
This would generate the following feedback:

Third example feedback

{"command": "start-judgement"}
{"title": "stdout", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "$ submission &lt;&lt;&lt; hello &gt; <a class=\"file-link\" target=\"_blank\">files_tests/hello_out.txt</a>", "format": "html"}, "command": "start-testcase"}
{"expected": "hello world!\n", "channel": "stdout", "command": "start-test"}
{"generated": "hello world!\n", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {"input_files": [{"path": "files_tests/hello_out.txt", "url": "media/hello_out.txt"}]}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

In the description it will use the special here-file syntax for stdin and output-redirection for stdout.
This description would normally appear if arguments were provided with stdin. The usage of stderr or stdout will now also indicate to not just show the content of stdin. Instead input- and output-redirection is applied. The information given in stdout will also be returned in the data section.

Fourth example

tabs:
- tab: stdout
  contexts:
  - testcases:
      - stdin: "hello"
        arguments: ["world"]
        stdout: 
           path: "files_tests/hello_out.txt"
           url: "media/hello_out.txt"

In the example above arguments are added. This provide the following feedback:

Fourth example feedback

{"command": "start-judgement"}
{"title": "stdout", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "$ submission world &lt;&lt;&lt; hello &gt; <a class=\"file-link\" target=\"_blank\">files_tests/hello_out.txt</a>", "format": "html"}, "command": "start-testcase"}
{"expected": "hello world!\n", "channel": "stdout", "command": "start-test"}
{"generated": "hello world!\n", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {"input_files": [{"path": "files_tests/hello_out.txt", "url": "media/hello_out.txt"}]}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

The output is almost the exact same. The only difference is the extra argument in description.

The same thing can also be done for stderr:

Fifth example

tabs:
- tab: stdout
  contexts:
  - testcases:
    - stdin:
        path: "hello.txt"
        url: "media/workdir/hello.txt"
      arguments: ["world"]
      stdout:
        path: "files_tests/hello_out.txt"
        content: "hello world!\n"
      stderr:
        path: "files_tests/hello_err.txt"
        data: "ERROR\n" # Deprecated

The content or data can also still be specified. This will mean that TESTed doesn't actually need to read the file from somewhere, but can still be shown in the feedback:

Fourth example feedback

{"command": "start-judgement"}
{"title": "stdout", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "$ submission world &lt; <a class=\"file-link\" target=\"_blank\">hello.txt</a> &gt; <a class=\"file-link\" target=\"_blank\">files_tests/hello_out.txt</a> 2&gt; <a class=\"file-link\" target=\"_blank\">files_tests/hello_err.txt</a>", "format": "html"}, "command": "start-testcase"}
{"expected": "ERROR\n", "channel": "stderr", "command": "start-test"}
{"generated": "", "status": {"enum": "wrong"}, "command": "close-test"}
{"expected": "hello world!\n", "channel": "stdout", "command": "start-test"}
{"generated": "hello world!\n", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {"input_files": [{"path": "files_tests/hello_out.txt", "content": "hello world!\n"}, {"path": "files_tests/hello_err.txt", "content": "ERROR\n"}, {"path": "hello.txt", "url": "media/workdir/hello.txt"}]}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

Now the description will also contain output-redirection for stderr. The description is also slightly different because stdin was provided with a file.

BrentBlanckaert · 2025-05-09T11:26:21Z

Documentation for new extensions (updated version)

This documentation will discuss all the changes made regarding the IO of a test.

Input files

Input files (currently files in TESTed) are a list of files defined in the DSL. Each file can be represented using two fields: path (required) and content. I will show a few examples to show what these do:

First example

tabs:
- tab: counting
  contexts:
  - testcases:
    - expression: count_words('fish.txt', 'sharks')
      return: 1
      input_files:
        - path: "fish.txt"
          content: "There are sharks in the water!"
        - path: "mammal.txt"
          content: "There are tigers in the water!"

A path must ALWAYS be provided. This is the string that will be shown to the student in the above expression. The path field (originally called name) contains relative path names and is relative to the workdir directory. In the case of expression, statement, or descriptions, an exact match will be checked. In the expression, that string becomes a hyperlink pointing to the provided content.

The presence of the content field also indicates that the file does not yet exist in the working directory. In this case, a file will still need to be created in the working directory.

The feedback for this example is the following:

First example feedback

{"command": "start-judgement"}
{"title": "counting", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "count_words(&#x27;<a class=\"file-link\" target=\"_blank\">fish.txt</a>&#x27;, &#x27;sharks&#x27;)", "format": "html"}, "command": "start-testcase"}
{"expected": "1", "channel": "return", "command": "start-test"}
{"generated": "1", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"message": {"description": "<div class='contains-file''><p>File: <a class=\"file-link\" target=\"_blank\"><span class=\"code\">mammal.txt</span></a></p></div>", "format": "html"}, "command": "append-message"}
{"data": {"statements": "count_words('fish.txt', 'sharks')", "files": [{"path": "fish.txt", "content": "There are sharks in the water!"}]}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

The seen input files are provided in the data and the start of the html to get make a hyperlink is provided in the description. Dodona will use to create their own hyperlink or pop-up.

Second example

tabs:
- tab: counting
  contexts:
  - testcases:
    - expression: count_words('fish.txt', 'sharks')
      return: 1
      input_files:
        - path: "fish.txt"

In this case, the file is expected to already be present in the working directory. Dodona will make the hyperlink in the description point to a file provided in the evaluation folder with the same path provided in path.
The feedback would be the following:

Second example feedback

{"command": "start-judgement"}
{"title": "counting", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "count_words(&#x27;<a class=\"file-link\" target=\"_blank\">fish.txt</a>&#x27;, &#x27;sharks&#x27;)", "format": "html"}, "command": "start-testcase"}
{"expected": "1", "channel": "return", "command": "start-test"}
{"generated": "1", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {"statements": "count_words('fish.txt', 'sharks')", "files": [{"path": "fish.txt"}]}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

The main difference is that only path is present in data.files.

Output files

In TESTed, you were only able to provide a single output file using file. This has been extended to support multiple.
An output_file contains both a path and a content field, both of which are required.

The path field specifies the path to the file that the student should have generated (in the working directory).
The content field contains either the expected contents of the generated file or a path to a real file containing the expected contents (located in the evaluation folder).

First example

tabs:
- tab: output_file
  contexts:
  - testcases:
    - expression: genereer('origineel_tekst.txt', 'text', 3)
      return: true
      output_files:
        - content: !path "files_tests/tekst1.txt"
          path: "text1.txt"
        - content: !path "files_tests/tekst2.txt"
          path: "text2.txt"
        - content: "Created using write mode.\n3\n"
          path: "text3.txt"
      input_files:
        - path: "origineel_tekst.txt"
          content: "Created using write mode.\n"

The feedback provided for this test would look something like the following:

{"command": "start-judgement"}
{"title": "output_file", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "genereer(&#x27;<a class=\"file-link\" target=\"_blank\">origineel_tekst.txt</a>&#x27;, &#x27;text&#x27;, 3)", "format": "html"}, "command": "start-testcase"}
{"expected": "Created using write mode.\n1\n", "channel": "file: text1.txt", "command": "start-test"}
{"generated": "Created using write mode.\n1\n", "status": {"enum": "correct"}, "command": "close-test"}
{"expected": "Created using write mode.\n2\n", "channel": "file: text2.txt", "command": "start-test"}
{"generated": "Created using write mode.\n2\n", "status": {"enum": "correct"}, "command": "close-test"}
{"expected": "Created using write mode.\n3\n", "channel": "file: text3.txt", "command": "start-test"}
{"generated": "Created using write mode.\n3\n", "status": {"enum": "correct"}, "command": "close-test"}
{"expected": "True", "channel": "return", "command": "start-test"}
{"generated": "True", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {"statements": "genereer('origineel_tekst.txt', 'text', 3)", "files": [{"path": "origineel_tekst.txt", "content": "Created using write mode.\n"}]}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

For each output file, the feedback includes a pair showing both the expected and the generated content. The channel field in the expected output will be labeled as file: <path of output file>. The full contents of both files are included directly in the feedback, as there is currently no support for returning files themselves instead of their contents.

Stdin

In TESTed, stdin could only be a string or any other basic type. I also expanded the capabilities for that.
If stdin is a string, it is equivalent to using the field content. Alternatively, a path (relative to the working directory) can be provided. If only path is provided with no content then it will be assumed that a file is present in the evaluation folder at the same location provided by path.

First example

tabs:
- tab: stdin
  contexts:
  - testcases:
      - stdin: "hello"
        stdout: "hello world!\n"

In this setup, stdin cannot be an object, as this would cause issues in the validator. It could be mistaken for one of the next examples, leading to incorrect interpretation or validation errors.

Second example

tabs:
- tab: stdin
  contexts:
  - testcases:
      - stdin:
          content: "hello"
        stdout: "hello world!\n"

This one is equivalent with the first example. The feedback for those would look something like this:

{"command": "start-judgement"}
{"title": "stdin", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "hello\n", "format": "console"}, "command": "start-testcase"}
{"expected": "hello world!\n", "channel": "stdout", "command": "start-test"}
{"generated": "hello world!\n", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

This is the exact same as it is now, but stdin is no longer present in data. This is because we didn't see any more use for it.

Third example

tabs:
- tab: stdin
  contexts:
  - testcases:
      - stdin: 
          path: "hello.txt"
        stdout: "hello world!\n"

In this case, the content must be read from a file. The output looks like this:

Third example feedback

{"command": "start-judgement"}
{"title": "stdin", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "$ submission &lt; <a class=\"file-link\" target=\"_blank\">hello.txt</a>", "format": "html"}, "command": "start-testcase"}
{"expected": "hello world!\n", "channel": "stdout", "command": "start-test"}
{"generated": "hello world!\n", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {"files": [{"path": "hello.txt"}]}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

The provided information about the file is also given in data.

Fourth example

tabs:
- tab: stdin
  contexts:
  - testcases:
      - stdin:
          path: "hello.txt"
          content: "hello"
        stdout: "hello world!\n"

In this case, the file doesn't need to physically exist in the working directory. The feedback will be the following:

Fourth example feedback

{"command": "start-judgement"}
{"title": "stdin", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "$ submission &lt; <a class=\"file-link\" target=\"_blank\">hello.txt</a>", "format": "html"}, "command": "start-testcase"}
{"expected": "hello world!\n", "channel": "stdout", "command": "start-test"}
{"generated": "hello world!\n", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {"files": [{"path": "hello.txt", "content": "hello\n"}]}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

Now the content will be provided in data.

Fifth example

tabs:
- tab: stdin
  contexts:
  - testcases:
      - stdin:
          path: "hello.txt"
          content: "hello"
        arguments: ["world"]
        stdout: "hello world!\n"

Because an argument was provided, the desciption will look a bit different:

Fifth example feedback

{"command": "start-judgement"}
{"title": "stdin", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "$ submission world &lt; <a class=\"file-link\" target=\"_blank\">hello.txt</a>", "format": "html"}, "command": "start-testcase"}
{"expected": "hello world!\n", "channel": "stdout", "command": "start-test"}
{"generated": "hello world!\n", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {"files": [{"path": "hello.txt", "content": "hello\n"}]}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

Sixth example

tabs:
- tab: stdin
  contexts:
  - testcases:
      - stdin:
          content: "hello"
        arguments: ["world"]
        stdout: "hello world!\n"

Sixth example feedback

{"command": "start-judgement"}
{"title": "stdin", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "$ submission world <<< hello", "format": "console"}, "command": "start-testcase"}
{"expected": "hello world!\n", "channel": "stdout", "command": "start-test"}
{"generated": "hello world!\n", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

TESTed uses here-files in this case, but for single-line content, you can use a special shorthand syntax, as shown in the description.

Seventh example

tabs:
- tab: stdin
  contexts:
  - testcases:
      - stdin:
          path: "hello.txt"
          content: "hello"
        arguments: ["world"]
        description: "stdin_test world < hello.txt"
        stdout: "hello world!\n"

Here the description will actually use the file provided in stdin to generate the start of a hyperlink:

Seventh example feedback:

{"command": "start-judgement"}
{"title": "stdin", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "stdin_test world &lt; <a class=\"file-link\" target=\"_blank\">hello.txt</a>", "format": "html"}, "command": "start-testcase"}
{"expected": "hello world!\n", "channel": "stdout", "command": "start-test"}
{"generated": "hello world!\n", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {"files": [{"path": "hello.txt", "content": "hello\n"}]}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

Stdout & Stderr

In TESTed, stdout and stderr already followed the exact same syntax. So it makes sense that that is still the case.
A few examples using stdout:

First example

tabs:
- tab: stdout
  contexts:
  - testcases:
      - stdin: "hello"
        stdout: "hello world!\n"

This was already possible in TESTed.

Second example

- tab: stdout
  contexts:
  - testcases:
      - stdin: "hello"
        stdout: 
          content: "hello world!\n"

In TESTed, the key data would be used. This is still possible, but content is a better name and more consistent with the new changes.

First and second example feedback

{"command": "start-judgement"}
{"title": "stdout", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "hello\n", "format": "console"}, "command": "start-testcase"}
{"expected": "hello world!\n", "channel": "stdout", "command": "start-test"}
{"generated": "hello world!\n", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

This is still the same as it is in TESTed.

Third example

tabs:
- tab: stdout
  contexts:
  - testcases:
      - stdin: "hello"
        stdout: 
           path: "files_tests/hello_out.txt"

Just like stdin, you can provide a path.
This would generate the following feedback:

Third example feedback

{"command": "start-judgement"}
{"title": "stdout", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "hello\n", "format": "console"}, "command": "start-testcase"}
{"expected": "hello world!\n", "channel": "stdout", "command": "start-test"}
{"generated": "hello world!\n", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

Currently the feedback will still be the same. In a later stage the new DSL could be used by Dodona to provide the feedback by file instead of just a string.

Fifth example

tabs:
- tab: stdout
  contexts:
  - testcases:
    - stdin:
        path: "hello.txt"
        url: "media/workdir/hello.txt"
      arguments: ["world"]
      stdout:
        path: "files_tests/hello_out.txt"
        content: "hello world!\n"
      stderr:
        path: "files_tests/hello_err.txt"
        data: "ERROR\n" # Deprecated

The content or data can still be specified directly. This means that TESTed doesn't need to read the file from an external location. While this isn't particularly useful at the moment, it will become more relevant in the future when Dodona supports including files in the feedback instead of displaying their full content directly.

Fourth example feedback

{"command": "start-judgement"}
{"title": "stdout", "command": "start-tab"}
{"command": "start-context"}
{"description": {"description": "$ submission world &lt; <a class=\"file-link\" target=\"_blank\">hello.txt</a>", "format": "html"}, "command": "start-testcase"}
{"expected": "ERROR\n", "channel": "stderr", "command": "start-test"}
{"generated": "ERROR\n", "status": {"enum": "correct"}, "command": "close-test"}
{"expected": "hello world!\n", "channel": "stdout", "command": "start-test"}
{"generated": "hello world!\n", "status": {"enum": "correct"}, "command": "close-test"}
{"command": "close-testcase"}
{"data": {"files": [{"path": "hello.txt"}]}, "command": "close-context"}
{"command": "close-tab"}
{"command": "close-judgement"}

jorg-vr · 2025-05-12T08:37:40Z

This version of the described features looks fine to me.
I would still remove the fifth example for stdout, as is has no implemented support for now. But that is not very important.

Is the code also ready for review?

BrentBlanckaert · 2025-05-12T08:40:40Z

Yes, the code is ready too.

BrentBlanckaert · 2025-05-12T08:45:16Z

Just realised that the way deprecated messages are done still need to change, but other than that it should be fine.

jorg-vr

I didn't review all json schema's, But I assume the feedback I have given on the strict schema is applicable to all

tested/descriptions/renderer.py

tested/dodona.py

tested/dsl/schema-strict.json

tested/oracles/text.py

jorg-vr · 2025-05-12T13:56:09Z

tested/testsuite.py

    type: TextChannelType = TextChannelType.TEXT

    def get_data_as_string(self, working_directory: Path) -> str:
        """Get the data as a string, reading the file if necessary."""
-        if self.type == TextChannelType.TEXT:
+        if self.data is not None:


This doesn't keep support for all deprecated usages (eg. with data and type TextChannelType.FILE)

This is used for stdin, stdout and stderr.

For stdin you could only provide a string. So, in this case nothing would change since data is the provided string and type is TextChannelType.TEXT.

For stdout and stderr, you could also only provide a string. So in this case there are no problems either.

Why would you still need to read the file if data is already provided?

this was the old code:

if self.type == TextChannelType.TEXT: return self.data elif self.type == TextChannelType.FILE: file_path = _resolve_path(working_directory, self.data) with open(file_path, "r") as file: return file.read()

So in the old case it was possible to have data and type TextChannelType.FILE.

Now as far as I understood data is deprecated and replaced by content and path

This check if self.data is not None: will return true if we are evaluating an old exercise making use of the deprecated data keyword.
Looking at the old code, this could mean we have to handle either TEXT or a FILE.
Your code acts as if the FILE case never existed.

So a couple of things could be happening here:

I completely misunderstood the code, the variable names here have a different meaning as those in the DSL for example...

In the old code elif self.type == TextChannelType.FILE was impossible to reach and never used, so we can remove it without issues

You need to add a check on self.type in the case self.data is not None

Could you tell me which of these three cases it is?

The variable data and the keys data and content in the DSL are the same thing here. This new check makes life easier because now you can provide content and path. In this case, there would be no need to read it from the file.
So I think you misunderstood the code.

However, I don't think there was any usecase for the type FILE before I made any changes.

tested/utils.py

tested/dsl/translate_parser.py

jorg-vr · 2025-05-12T14:07:16Z

tested/dsl/translate_parser.py

@@ -148,6 +168,7 @@ def _parse_yaml(yaml_stream: str) -> YamlObject:
            yaml.add_constructor("!" + actual_type, _custom_type_constructors, loader)
    yaml.add_constructor("!expression", _expression_string, loader)
    yaml.add_constructor("!oracle", _return_oracle, loader)
+    yaml.add_constructor("!path", _path_string, loader)


has this ever been discussed during one of your thesis meetings?

BrentBlanckaert · 2025-05-12T16:47:02Z

has this ever been discussed during one of your thesis meetings?

It has. This is used in the output_files and all of us were fine with the usage and the name.

tested/judge/evaluation.py

BrentBlanckaert added 2 commits January 11, 2025 19:45

changed some names

99a26c5

fixed some tests

667fd20

BrentBlanckaert mentioned this pull request Dec 20, 2024

Planned extensions for Brent's master's thesis #559

Open

67 tasks

BrentBlanckaert self-assigned this Jan 11, 2025

BrentBlanckaert added the enhancement New feature or request label Jan 11, 2025

BrentBlanckaert added 13 commits January 12, 2025 11:34

fixed a test

4a07970

have something that works for stdin

e834e9f

have something working for stdout/stderr

791836e

fixed all tests

ef29394

fixed some small issues

a00f09e

made something that can be used to generate and evaluate multiple files.

ee2123c

Updated output files to be more consistent and do better check. Also …

1b3b892

…make stdin more consistent

Made inlining for stdout and stderr possible and made everything more…

32bb7d9

… consitent

Fixed some minor issues

2afdd67

Fixed the dsl yaml tests

e01ebdc

Fixed buildin oracle tests

2ce7892

Fixed the io function

3a62f6d

Fixed linting issue

a845fb2

BrentBlanckaert added 8 commits February 18, 2025 20:53

fixed last tests

e939f30

Added an extra test

10bd418

Added some more tests

1d3aebd

lint test

2cb31f5

covered an extra case

2ce6701

added an extra case for invalid input

70e3f21

fixed test

441fc8b

fixed linting

9254c14

BrentBlanckaert marked this pull request as ready for review February 25, 2025 16:20

BrentBlanckaert requested review from niknetniko and jorg-vr February 25, 2025 16:20

BrentBlanckaert requested a review from jorg-vr April 20, 2025 15:22

BrentBlanckaert added 3 commits April 25, 2025 19:14

slight name change

6a0808d

updated input_files

5c9a389

updated stdin, stdout and stderr

858819d

BrentBlanckaert added 5 commits April 26, 2025 16:37

fixed test

07c57ca

input files no longer use url

80ff847

small update for output files

fd15d8c

removed url from stdin

f440484

changed stdout en stderr

5599053

BrentBlanckaert added 3 commits May 9, 2025 13:39

fixed tests

1cab274

Made an extra test

aab198e

Fixed linting

2b90afd

changed how deprecated messages are handled

2c06c20

jorg-vr requested changes May 12, 2025

View reviewed changes

solved some of the suggested issues

04c28ad

github-advanced-security bot found potential problems May 12, 2025

View reviewed changes

tested/judge/evaluation.py Fixed Show fixed Hide fixed

BrentBlanckaert added 4 commits May 12, 2025 18:55

remove weird import added by IDE

8cdd78b

fixed tests

ef53242

did a few simple changes

77d78c7

fixed linting

b9d168b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add file content #577

add file content #577

BrentBlanckaert commented Jan 11, 2025

BrentBlanckaert commented Feb 18, 2025 •

edited

Loading

jorg-vr commented Apr 25, 2025

BrentBlanckaert commented Apr 25, 2025

jorg-vr commented Apr 25, 2025

BrentBlanckaert commented Apr 25, 2025

BrentBlanckaert commented Apr 26, 2025

BrentBlanckaert commented May 9, 2025

jorg-vr commented May 12, 2025

BrentBlanckaert commented May 12, 2025

BrentBlanckaert commented May 12, 2025

jorg-vr left a comment

jorg-vr May 12, 2025

BrentBlanckaert May 12, 2025

jorg-vr May 13, 2025

BrentBlanckaert May 13, 2025

jorg-vr May 12, 2025

BrentBlanckaert commented May 12, 2025

add file content #577

Are you sure you want to change the base?

add file content #577

Conversation

BrentBlanckaert commented Jan 11, 2025

BrentBlanckaert commented Feb 18, 2025 • edited Loading

Documentation for usage of files

Files

Stdin, Stdout and Stderr

jorg-vr commented Apr 25, 2025

Input files

First example

First example feedback

Second example

some more feedback

Output files

First example

Stdin

first example

second example

Third example

Fourth example

Fifth example

sixth example

Seventh example

Stdout & Stderr

First example

Second example

Third example

Fourth example

Fifth example

BrentBlanckaert commented Apr 25, 2025

Input files

First example

Second example

Output files

Stdin

Third example

Seventh example

Stdout & Stderr

Third example

Fourth example

Fifth example

jorg-vr commented Apr 25, 2025

BrentBlanckaert commented Apr 25, 2025

BrentBlanckaert commented Apr 26, 2025

Documentation for new extensions (updated version from feedback)

Input files

First example

First example feedback

Second example

Second example feedback

Output files

First example

Stdin

First example

Second example

Third example

Third example feedback

Fourth example

Fourth example feedback

Fifth example

Fifth example feedback

Sixth example

Sixth example feedback

Seventh example

Seventh example feedback:

Stdout & Stderr

First example

Second example

First and second example feedback

Third example

Third example feedback

Fourth example

Fourth example feedback

Fifth example

Fourth example feedback

BrentBlanckaert commented May 9, 2025

Documentation for new extensions (updated version)

Input files

BrentBlanckaert commented Feb 18, 2025 •

edited

Loading