#rustlang #csv #rust #arrow
@veer66 you don’t have to. Which csv crate are you using, and did you use serde’s macros to map the header fields to whatever you wanted them to be? Extra stuff doesn’t matter.
@veer66 you can use Option<T> to make your destination field optional on read, and then add the value as Some<T> before you write it. Make sure you read the csv into a mutable struct and you should be good to go.
@veer66 good deal. I generally turn to Python when my Rust chops aren’t enough. If Apache Arrow handles your use case rock on with it!
@veer66 I know the feeling. One of my Advent of Code solutions that disgusts a real software engineer for its inefficient algorithm still *finishes execution* before Python can even launch its interpreter. Still, Python remains a useful tool on which I have created quite a few mission critical tools.
Last week I processed my dataset and I switched to Rust when Python version cloud't finish the task in six hours, while Rust version took 22 seconds. Later Sorrawut Kittikeereechaikun showed me that I can avoid converting between Arrow/Numpy numbers to Python numbers too often. Then Python version can take only 4 times slower than Rust version.
I mean Python is very useful and not too slow for most of my use-cases. Rust helps me getting a long task done without being clever.
@veer66 That's true, but unless you're running a daemon that interpreter startup time is per execution and the build cost (for a given version) happens exactly once. Of course I think this is amplified on simpler problems where the comparison is fairly simple std lib vs std lib work. Otherwise like most things it's about the right tool for the job, and the compile time certainly factors in!
