Friday, February 12, 2016

How not to use awk

Today I was working on automating AMI creation for a project and ended up using packer. So packer has this thing called -machine-readable output which can be easily parsed as CSV (at the very least). I ended up writing a bash script to parse the output.
I started off with using while loop + awk for parsing (in that order) and emitting the lines that contains the artifact information. As you can see above for parsing around 1700+ lines it took > 6 seconds.

Afterwards I refactored the code to use awk first and feed that output to while loop, which actually ran 100x faster.

I didn't knew AWK was so good at processing things at scale (if I may). 

No comments:

Post a Comment