包含.csv中变量的散列行的java解决方案

I have a file that represent a table recorded in .csv or similar format. Table may include missing values. I look for a solution (preferably in java), that would process my file in the incremental manner without loading everything into memory, as my file can be huge. I need to identify duplicate records in my file, being able to specify which columns I want to exclude from consideration; then produce an output grouping those duplicate records. I would add an additional value at the end with a group number and output in the same format (.csv) sorted by group number.I have a file that represent a table recorded i