Skip to content

Add csimdv to count benchmark#55

Open
juliusgeo wants to merge 2 commits intomedialab:masterfrom
juliusgeo:add_csimdv_example
Open

Add csimdv to count benchmark#55
juliusgeo wants to merge 2 commits intomedialab:masterfrom
juliusgeo:add_csimdv_example

Conversation

@juliusgeo
Copy link

@juliusgeo juliusgeo commented Mar 17, 2026

Adds csimdv to benchmark file.
On my aarch64 machine, it works without any rustflags. On x86_64 (Windows), I had to provide RUSTFLAGS="target-cpu=native".

@Yomguithereal
Copy link
Member

First, on my end I need RUSTFLAGS="-C target-feature=+pclmulqdq,+avx2" because I don't have avx512f.

@Yomguithereal
Copy link
Member

Then, my functional tests crash because your buffer here: https://github.com/juliusgeo/csimdv-rs/blob/master/src/constants.rs#L3 is too short for real-life CSV data (any file containing raw text in a cell, like press articles, will overflow).

@Yomguithereal
Copy link
Member

Now with the files I can test (the ones which don't overflow the buffer), c-simd-v is consistently slower than the copying reader. Let me try an aggregate stats better. I must also find a CPU with AVX-512, since I am only benchmarking AVX2 here.

@juliusgeo
Copy link
Author

juliusgeo commented Mar 18, 2026

@Yomguithereal I bumped the buffer size in csimdv to 1mb, which should (hopefully) be large enough to accommodate real world files. I also realized that in the benchmark I was not only counting the number of rows in the CSV file, but also the length of each cell in the rows. I ran the benchmarks from this repo on the examples from the csimdv repo, and the results align much better after that change. Seeing roughly the same speed from zero-copy and csimdv on aarch64, will have to test later on AVX-512.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants