I understand you want to jot down the unique records and redirect them to another file.

There is command called uniq in unix which works on sorted data input. You could use the uniq command to eliminate or count duplicate lines. It works by comparing the previous and current line.

Let take your csv file to be duplicates.csv

Now consider the follwing:

cat duplicates|sort | uniq -c > unique.csv

The above would give the unique records along with the count of the number of times it occurred in the input.

using -d instead of -c would deletes duplicates and give only unique records.

If you want to work on a particular field/column only then you could use a combination of  cut and uniq commands.



Hope this helps you. I could elaborate more.