Skip to content

Submission Analyzer

A central component our study investigation was the assessment of code produced by study participants. Given the amount of code to analyze, we wrote an automated tool, to test the submissions against a stipulated target REST interface. Source code and documentation are available as open source on GitHub

How it works

  • Download the raw participant code submissions.
  • Clone the analyzer tool and follow the run instructions it. You will obtain a CSV test report with details on every participant submission.
  • You can match the test reports against the data we used for our statistical analysis and paper figures. It serves as starting point for the result replication.