Arete - automated testing service

Services

Monitoring service

Graphical user interface for monitoring and managing Arete.

Authentication Service

Authentication service authenticates, saves and caches everything to do with Arete.

Automated Testing Service

Service which summons and handles docker containers for student code testing.

Backend is proxied by cs.ttu.ee.

Source code is in gitlab.

Test Runners

Custom test runner which usually runs student code against tests.
Can also make an API request against external testing service like Online Judge.

Gitlab group with source code

Class structure of objects transferred between services

All the class structures can be viewed at https://cs.ttu.ee/services/arete/api/v2/swagger-ui/

Client is available at github

Flows

Updating tests

test update

Running tests

running tests

Running Locally

If you wish to run arete locally or on a server, a special docker-compose file has been created for that purpose: docker-compose-arete.yml
Along with the required nginx.conf file

The nginx service will facilitate the connection between the monitoring service and the authentication service as to not cause any CORS errors.

All environment variables have defaults provided except GITLAB_REGISTRY_USER and GITLAB_REGISTRY_TOKEN, which need to be put in a .env file
If this is set up on a server NGINX_BACKEND will need to be updated, so it has the same host as the server.

Variables can be set in the .env file

Scripts

To make self-hosting easier on Windows and Linux devices, scripts have been created to help out

The Scripts handle everything from downloading the required programs and files to starting the docker containers.

Type ./arete.sh help or ./arete.ps1 help for help

If you have any issues with the scripts, please email ssaart@ttu.ee

By default, admin username and password will be admin and admin respectively.

It can be configured in the .env file along with other passwords, urls and ports.

Steps to Creating a submission

  1. In the monitoring service dashboard at /services/arete/admin,
    create a user with the role TESTER and the credentials of BACKEND_TOKEN (username and token are seperated by a space).

  2. Create 2 git repositories. One for tests and the other for user/student files, or use existing ones.

  3. Update tests at /services/arete/submissions with the test repository.

  4. Fill out the created submission form with the needed info (testing language, student repository, tester repository, UNI-ID, Testing Mode)

Running integration tests

NOTE: integration tests require access to bunch of repositories in GitLab

  • https://gitlab.cs.ttu.ee/groups/yfx0505-2021/-/group_members

  • https://gitlab.cs.ttu.ee/groups/iti0102-2018/-/group_members

  • https://gitlab.cs.ttu.ee/groups/iti0102-2020/-/group_members

  • https://gitlab.cs.ttu.ee/groups/iti0202-2019/-/group_members

  • https://gitlab.cs.ttu.ee/groups/iti0211-2019/-/group_members

version: '3.4'

services:

    arete-test:
        image: gitlab.cs.ttu.ee:5050/testing/arete-integration-tests:latest
        environment:
            AUTH_CREDENTIALS: ${AUTH_CREDENTIALS:-admin admin}
            AUTHENTICATION_SERVICE_URL: ${AUTHENTICATION_SERVICE_URL:-http://127.0.0.1:8001/services/arete/api/v2}
            LOAD_BALANCER_URL: ${LOAD_BALANCER_URL:-http://127.0.0.1:8098}

Some useful curl commands to test arete whether it's working correctly

# Force an update on a test:
curl -d '{"project":{"url":"git@gitlab.cs.ttu.ee:iti0102-2019/ex.git","path_with_namespace":"iti0211-2019/ex"}}' -X POST -H "Content-Type: application/json" localhost:8098/tests:update

# Force an update on a image
curl -X POST -H "Content-Type: application/json" localhost:8098/image/python-tester:update

# Test something sync
curl -d '{"testingPlatform":"prolog","gitTestSource":"https://gitlab.cs.ttu.ee/iti0211-2019/tests","gitStudentRepo":"https://gitlab.cs.ttu.ee/envomp/iti0211-2019.git","uniid":"envomp"}' -X POST -H "Content-Type: application/json" localhost:8098/:testSync

cleanup (kill all containers, networks, volumes, images)

docker rm -vf $(docker ps -aq) &&
  docker rmi -f $(docker images -aq) &&
  docker volume prune -f