Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When running scarb test or scarb cairo-test with the --json flag, the test output should be in JSON format #1474

Open
0xNeshi opened this issue Jul 24, 2024 · 0 comments

Comments

@0xNeshi
Copy link

0xNeshi commented Jul 24, 2024

Problem

The --json flag should result in all output being in JSON format. This would be extremely helpful when creating a Test Runner for the Cairo track on Exercism.

Currently this flag does virtually nothing to format the test output:

  • scarb --json cairo-test --include-ignored just prints everything in multiple lines:
    image
  • scarb --json test --include-ignored prints the current operation as JSON, and then prints the actual test output the same as before:
    image

Proposed Solution

The output for the tests should be in JSON format, something like:

{"status":"compiling","message":"test(leap_unittest) leap v0.1.0 (/home/nenad/repos/work/cairo-test-runner/tests/partial-fail/Scarb.toml)"}
{"status":"compiling","message":"test(leap_leap) leap_leap v0.1.0 (/home/nenad/repos/work/cairo-test-runner/tests/partial-fail/Scarb.toml)"}
{"status":"finished","message":"release target(s) in 2 seconds"}
{"status":"running","message":"cairo-test leap"}
{"status":"running","message":"9 tests"}
{"status":"ok","test":"leap_leap::leap::year_not_divisible_by_4_in_common_year", "gas_usage_est": 46040}
{"status":"ok","test":"leap_leap::leap::year_divisible_by_400_but_not_by_125_is_still_a_leap_year", "gas_usage_est": 45840}
{"status":"fail","test":"leap_leap::leap::year_divisible_by_200_not_divisible_by_400_in_common_year", "gas_usage_est": 50580}
{"status":"fail","test":"leap_leap::leap::year_divisible_by_100_not_divisible_by_400_in_common_year", "gas_usage_est": 50580}
{"status":"ok","test":"leap_leap::leap::year_divisible_by_400_is_leap_year", "gas_usage_est": 45840}
{"status":"ok","test":"leap_leap::leap::year_divisible_by_4_not_divisible_by_100_in_leap_year", "gas_usage_est": 45840}
{"status":"ok","test":"leap_leap::leap::year_divisible_by_2_not_divisible_by_4_in_common_year", "gas_usage_est": 46040}
{"status":"ok","test":"leap_leap::leap::year_divisible_by_4_and_5_is_still_a_leap_year", "gas_usage_est": 45840}
{"status":"fail","test":"leap_leap::leap::year_divisible_by_100_but_not_by_3_is_still_not_a_leap_year", "gas_usage_est": 50580}
{"status":"finished", "failures": [
    {"test": "leap_leap::leap::year_divisible_by_200_not_divisible_by_400_in_common_year", "message": "Panicked with \"assertion failed: `!is_leap_year(1800)`.\""},
    {"test": "leap_leap::leap::year_divisible_by_100_not_divisible_by_400_in_common_year", "message": "Panicked with \"assertion failed: `!is_leap_year(2100)`.\""},
    {"test": "leap_leap::leap::year_divisible_by_100_but_not_by_3_is_still_not_a_leap_year", "message": "Panicked with \"assertion failed: `!is_leap_year(1900)`.\""},
]}
{"status":"finished", "message":"Error: test result: FAILED. 6 passed; 3 failed; 0 ignored"}

Notes

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Triage
Development

No branches or pull requests

1 participant