Today I was working on automating AMI creation for a project and ended up using packer. So packer has this thing called -machine-readable output which can be easily parsed as CSV (at the very least). I ended up writing a bash script to parse the output.
I started off with using while loop + awk for parsing (in that order) and emitting the lines that contains the artifact information. As you can see above for parsing around 1700+ lines it took > 6 seconds.
Afterwards I refactored the code to use awk first and feed that output to while loop, which actually ran 100x faster.
I didn't knew AWK was so good at processing things at scale (if I may).
I started off with using while loop + awk for parsing (in that order) and emitting the lines that contains the artifact information. As you can see above for parsing around 1700+ lines it took > 6 seconds.
Afterwards I refactored the code to use awk first and feed that output to while loop, which actually ran 100x faster.
I didn't knew AWK was so good at processing things at scale (if I may).
No comments:
Post a Comment