Answer by David C. Rankin for Remove duplicated lines
You can do it in a single pass with a short script solution in bash utilizing arrays. (depending on the length of the file, you may find a double pass with awk faster, but for larger files, this...
View ArticleAnswer by Andreas Louv for Remove duplicated lines
By doing two iterations over the log file, you can collect all used dates in the first iteration. And then remove NULL logs if the date is already used in the second iteration:$ awk -F, '$4$5 !~ /NULL/...
View ArticleAnswer by 123 for Remove duplicated lines
You could use this awk.Iterate over the file twice.Count dupes first time.Print line that aren't dupes or don't contain NULL the second. awk -F, 'NR==FNR{a[$1]++;next}a[$1]<2||$4$5!~/NULL/'...
View ArticleRemove duplicated lines
I need some help with transforming broken logs, I receive from my admins (unfortunately they not gonna fix it).Basically, sometimes log contains duplicated first column, that I want to remove if...
View Article
