Add dynamic layout basic support

[JulianGCalderon]

Add dynamic layout support

This PR doesn't change any logic in the VM, it only changes some signatures to allow for dynamic layout parameters. The missing functionality is being developed in #1838.

It also adds a new job to the CI for testing dynamic layouts in different scenarios.

I separated it this way to make reviewing easier.

Description

This PR adds support for dynamic layout. When using dynamic layout, the actual parameters of the layout must be specified through a path to a JSON file in the command line arguments:

cairo-compile cairo_programs/abs_value_array.cairo --output cairo_programs/abs_value_array_compiled.json

target/release/cairo-vm-cli cairo_programs/abs_value_array_compiled.json \
    --layout dynamic \
    --cairo_layout_params_file cairo_layout_params_file.example.json

Or, with Cairo 1:

cargo run --bin cairo1-run cairo_programs/cairo-1-programs/array_get.cairo \
    --layout dynamic \
    --cairo_layout_params_file cairo_layout_params_file.example.json

To add this functionality, the public API had to be changed. Now both CairoRunner::new and CairoRunner::new_v2 receive as their 3rd argument an Option<CairoLayoutParams>. This must be Some only when using dynamic layout. Most changes in this PR are a side effect of this modification.

impl CairoRunner {
    /// The `cairo_layout_params` argument should only be used with dynamic layout.
    /// It is ignored otherwise.
    pub fn new_v2(
        program: &Program,
        layout: LayoutName,
        cairo_layout_params: Option<CairoLayoutParams>,
        mode: RunnerMode,
        trace_enabled: bool,
    ) -> Result<CairoRunner, RunnerError> {

Even though this is not the most "rustic" solution, it was done this way to facilitate usage and minimize changes needed.

The JSON file has the following format. I left an example in the repository's root just in case, it can be moved or removed.

{
    "rc_units": 4,
    "log_diluted_units_per_step": 4,
    "cpu_component_step": 8,
    "memory_units_per_step": 8,
    "uses_pedersen_builtin": true,
    "pedersen_ratio": 256,
    "uses_range_check_builtin": true,
    "range_check_ratio": 8,
    "uses_ecdsa_builtin": true,
    "ecdsa_ratio": 2048,
    "uses_bitwise_builtin": true,
    "bitwise_ratio": 16,
    "uses_ec_op_builtin": true,
    "ec_op_ratio": 1024,
    "uses_keccak_builtin": true,
    "keccak_ratio": 2048,
    "uses_poseidon_builtin": true,
    "poseidon_ratio": 256,
    "uses_range_check96_builtin": true,
    "range_check96_ratio": 8,
    "range_check96_ratio_den": 1,
    "uses_add_mod_builtin": true,
    "add_mod_ratio": 128,
    "add_mod_ratio_den": 1,
    "uses_mul_mod_builtin": true,
    "mul_mod_ratio": 256,
    "mul_mod_ratio_den": 1
}

Checklist

• Linked to Github Issue
• Unit tests added
• Integration tests added.
• This change requires new documentation.
  • Documentation has been added/updated.
  • CHANGELOG has been updated.

[github-actions]

**Hyper Thereading Benchmark results**




hyperfine -r 2 -n "hyper_threading_main threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_main' -n "hyper_threading_pr threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 1
  Time (mean ± σ):     30.115 s ±  0.027 s    [User: 29.351 s, System: 0.762 s]
  Range (min … max):   30.096 s … 30.134 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 1
  Time (mean ± σ):     30.559 s ±  0.234 s    [User: 29.746 s, System: 0.811 s]
  Range (min … max):   30.393 s … 30.724 s    2 runs
 
Summary
  'hyper_threading_main threads: 1' ran
    1.01 ± 0.01 times faster than 'hyper_threading_pr threads: 1'




hyperfine -r 2 -n "hyper_threading_main threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_main' -n "hyper_threading_pr threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 2
  Time (mean ± σ):     16.877 s ±  0.045 s    [User: 29.729 s, System: 0.819 s]
  Range (min … max):   16.845 s … 16.908 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 2
  Time (mean ± σ):     16.908 s ±  0.013 s    [User: 29.770 s, System: 0.809 s]
  Range (min … max):   16.898 s … 16.917 s    2 runs
 
Summary
  'hyper_threading_main threads: 2' ran
    1.00 ± 0.00 times faster than 'hyper_threading_pr threads: 2'




hyperfine -r 2 -n "hyper_threading_main threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_main' -n "hyper_threading_pr threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 4
  Time (mean ± σ):     11.384 s ±  0.078 s    [User: 41.635 s, System: 0.972 s]
  Range (min … max):   11.329 s … 11.439 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 4
  Time (mean ± σ):     11.467 s ±  0.133 s    [User: 41.516 s, System: 1.003 s]
  Range (min … max):   11.373 s … 11.562 s    2 runs
 
Summary
  'hyper_threading_main threads: 4' ran
    1.01 ± 0.01 times faster than 'hyper_threading_pr threads: 4'




hyperfine -r 2 -n "hyper_threading_main threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_main' -n "hyper_threading_pr threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 6
  Time (mean ± σ):     11.699 s ±  0.183 s    [User: 41.256 s, System: 1.006 s]
  Range (min … max):   11.569 s … 11.828 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 6
  Time (mean ± σ):     11.229 s ±  0.176 s    [User: 42.007 s, System: 1.010 s]
  Range (min … max):   11.104 s … 11.353 s    2 runs
 
Summary
  'hyper_threading_pr threads: 6' ran
    1.04 ± 0.02 times faster than 'hyper_threading_main threads: 6'




hyperfine -r 2 -n "hyper_threading_main threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_main' -n "hyper_threading_pr threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 8
  Time (mean ± σ):     11.150 s ±  0.036 s    [User: 42.036 s, System: 1.033 s]
  Range (min … max):   11.124 s … 11.175 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 8
  Time (mean ± σ):     11.422 s ±  0.110 s    [User: 42.391 s, System: 0.980 s]
  Range (min … max):   11.345 s … 11.500 s    2 runs
 
Summary
  'hyper_threading_main threads: 8' ran
    1.02 ± 0.01 times faster than 'hyper_threading_pr threads: 8'




hyperfine -r 2 -n "hyper_threading_main threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_main' -n "hyper_threading_pr threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 16
  Time (mean ± σ):     11.130 s ±  0.019 s    [User: 42.280 s, System: 1.116 s]
  Range (min … max):   11.117 s … 11.144 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 16
  Time (mean ± σ):     11.398 s ±  0.293 s    [User: 42.719 s, System: 1.102 s]
  Range (min … max):   11.191 s … 11.605 s    2 runs
 
Summary
  'hyper_threading_main threads: 16' ran
    1.02 ± 0.03 times faster than 'hyper_threading_pr threads: 16'

[github-actions]

Benchmark Results for unmodified programs 🚀

Command	Mean [s]	Min [s]	Max [s]	Relative
base big_factorial	2.383 ± 0.019	2.363	2.426	1.00
head big_factorial	2.430 ± 0.017	2.411	2.463	1.02 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base big_fibonacci	2.352 ± 0.060	2.316	2.517	1.00
head big_fibonacci	2.377 ± 0.027	2.353	2.441	1.01 ± 0.03

[codecov]

Codecov Report

Attention: Patch coverage is 94.84536% with 15 lines in your changes missing coverage. Please review.

Project coverage is 94.85%. Comparing base (5f8a011) to head (47e441b).

Files with missing lines	Patch %	Lines
vm/src/types/layout.rs	93.26%	13 Missing ⚠️
vm/src/cairo_run.rs	75.00%	1 Missing ⚠️
vm/src/vm/runners/cairo_runner.rs	94.11%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1824      +/-   ##
==========================================
+ Coverage   94.83%   94.85%   +0.01%     
==========================================
  Files         101      101              
  Lines       39579    39828     +249     
==========================================
+ Hits        37536    37779     +243     
- Misses       2043     2049       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

Benchmark Results for modified programs 🚀

Command	Mean [s]	Min [s]	Max [s]	Relative
head blake2s_integration_benchmark	8.883 ± 0.399	8.450	9.581	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
head compare_arrays_200000	2.485 ± 0.037	2.454	2.576	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
head dict_integration_benchmark	1.670 ± 0.030	1.650	1.728	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
head field_arithmetic_get_square_benchmark	1.384 ± 0.038	1.360	1.490	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
head integration_builtins	8.681 ± 0.174	8.534	9.001	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
head keccak_integration_benchmark	9.072 ± 0.255	8.832	9.618	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
head linear_search	2.496 ± 0.039	2.469	2.605	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
head math_cmp_and_pow_integration_benchmark	1.729 ± 0.037	1.714	1.834	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
head math_integration_benchmark	1.699 ± 0.057	1.672	1.859	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
head memory_integration_benchmark	1.400 ± 0.013	1.390	1.436	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
head operations_with_data_structures_benchmarks	1.779 ± 0.006	1.770	1.789	1.00

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
head pedersen	579.6 ± 4.8	573.5	589.7	1.00

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
head poseidon_integration_benchmark	683.7 ± 3.5	676.0	690.3	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
head secp_integration_benchmark	2.011 ± 0.014	1.985	2.029	1.00

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
head set_integration_benchmark	705.7 ± 7.5	700.8	725.6	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
head uint256_integration_benchmark	4.818 ± 0.065	4.749	4.933	1.00