How to remove duplicates from the list?

Question

I want to remove duplicates from the list, but what I am doing does not work:

List<Customer> listCustomer = new ArrayList<Customer>(); for (Customer customer: tmpListCustomer) { if (!listCustomer.contains(customer)) { listCustomer.add(customer); } } 

Answer 1

If this code does not work, you probably have not implemented equals(Object) in the Customer class.

Presumably there is some key (let's call it customerId ) that uniquely identifies the client; eg.

 class Customer { private String customerId; ... 

The corresponding definition for equals(Object) will look like this:

  public boolean equals(Object obj) { if (obj == this) { return true; } if (!(obj instanceof Customer)) { return false; } Customer other = (Customer) obj; return this.customerId.equals(other.customerId); } 

For completeness, you must also implement hashCode so that two identical Customer objects return the same hash value. The corresponding hashCode for the above equals definition would be:

  public int hashCode() { return customerId.hashCode(); } 

It is also worth noting that this is not an effective way to remove duplicates if the list is large. (For a list with N clients, you need to perform N*(N-1)/2 comparisons in the worst case, i.e. when there are no duplicates.) For a more efficient solution, you should use something like a HashSet to check for duplicates.

Answer 2

Assuming you want to keep the current order and don't want Set , perhaps the easiest way:

 List<Customer> depdupeCustomers = new ArrayList<>(new LinkedHashSet<>(customers)); 

If you want to change the original list:

 Set<Customer> depdupeCustomers = new LinkedHashSet<>(customers); customers.clear(); customers.addAll(dedupeCustomers); 

Answer 3

Does the client equals() contract?

If it does not implement equals() and hashCode() , then listCustomer.contains(customer) checks if the same instance exists in the list (for example, I mean the exact address of the object - memory, etc.). If what you are looking for is to check if there is the same Client in this list (possibly the same client, if they have the same client name or client number), then you will need to override equals() , to ensure that it checks to see if the corresponding fields (for example, customer names) match.

Note. Remember to override hashCode() if you are going to override equals() ! Otherwise, you may have problems with your HashMaps and other data structures. For a good coverage of the reasons for this and what can be avoided, consider looking at the Josh Bloch chapters Effective Java on equals() and hashCode() (The link contains only information on why you should implement hashCode() when implementing equals() , but there is a good idea on how to override equals() too).

By the way, is there a limit on your set? If this does not happen, it is a little easier to solve this problem, use Set<Customer> like this:

 Set<Customer> noDups = new HashSet<Customer>(); noDups.addAll(tmpListCustomer); return new ArrayList<Customer>(noDups); 

It will be nice to delete duplicates for you, since the sets do not allow duplicates. However, this will lose any ordering that has been applied to tmpListCustomer , since the HashSet does not have an explicit ordering (you can get around this with the TreeSet , but this is not entirely related to your question). This may simplify your code a bit.

Answer 4

Java update 8
you can use array stream as below:

 Arrays.stream(yourArray).distinct() .collect(Collectors.toList()); 

Answer 5

List → Set → List (separate)

Just add all your elements to Set : it does not allow you to repeat its elements. If after that you need a list, use the new ArrayList(theSet) constructor (where theSet is your result set).

Answer 6

I suspect that you may not have Customer.equals() correctly implemented (or generally).

List.contains() uses equals() to check if any of its elements are identical to the object passed as a parameter. However, the default implementation of equals checks the physical identity, not the value identifier. Therefore, if you did not overwrite it in Customer , it will return false for two different Customer objects that have the same state.

Here are the detailed details of how to implement equals (and hashCode , which is its pair - you should almost always implement both options if you need to implement either of them). Since you have not shown us the Customer class, it is difficult to give more specific recommendations.

As others have noted, you'd better use Set instead of doing the job manually, but even for this you still need to implement these methods.

Answer 7

The contains method checks to see if the list contains an entry that returns true from Customer.equals (Object o). If you did not redefine equals (Object) in Customer or one of its parents, then it will only look for an existing occurrence of the same object. Perhaps this was what you wanted, in which case your code should work. But if you were looking for not having two objects representing the same client, then you need to override equals (Object) to return true when that is.

It’s also true that using one of the Set implementations instead of List would give you automatic deletion automatically and faster (for anything but very small lists). You still need to provide the code for equals.

You must also override hashCode () when overriding equals ().

Answer 8

 private void removeTheDuplicates(List<Customer>myList) { for(ListIterator<Customer>iterator = myList.listIterator(); iterator.hasNext();) { Customer customer = iterator.next(); if(Collections.frequency(myList, customer) > 1) { iterator.remove(); } } System.out.println(myList.toString()); } 

Answer 9

Two suggestions:

• Use a HashSet instead of an ArrayList. This will greatly speed up the contains () check if you have a long list

• Ensure that Customer.equals () and Customer.hashCode () are implemented correctly, that is, they must be based on the combined values ​​of the underlying fields in the client object.

Answer 10

Almost all of the answers above are correct, but I suggest using Map or Set when creating the appropriate list, and not after getting the performance. Since converting the list to Set or Map and then converting it to the list again is trivial work.

Code example:

 Set<String> stringsSet = new LinkedHashSet<String>();//A Linked hash set //prevents the adding order of the elements for (String string: stringsList) { stringsSet.add(string); } return new ArrayList<String>(stringsSet); 

Answer 11

As already mentioned, you probably are not using equals () correctly.

However, it should also be noted that this code is considered very inefficient, since the execution time may be the number of squares of the elements.

You might want to consider using the Set structure instead of a list, or create a set first and then turn it into a list.

Answer 12

The cleanest way:

 List<XXX> lstConsultada = dao.findByPropertyList(YYY); List<XXX> lstFinal = new ArrayList<XXX>(new LinkedHashSet<GrupoOrigen>(XXX)); 

and override hascode and equals by Id properties of each object

Answer 13

IMHO the best way to do it these days:

Suppose you have a collection of " dups " and you want to create another collection containing the same elements, but with all the duplicate exceptions. The following single-line trick does the trick.

 Collection<collectionType> noDups = new HashSet<collectionType>(dups); 

It works by creating a collection that, by definition, cannot contain duplicates.

Based on oracle document. A.

Answer 14

The correct answer for Java is Set . If you already have a List<Customer> and want to duplicate it

 Set<Customer> s = new HashSet<Customer>(listCustomer); 

Otherwise, just use the Set implementation of HashSet , TreeSet and skip the List construction step.

You need to override hashCode() and equals() in your domain classes, which are placed in Set , and also make sure that the behavior you want is really what you get. equals() can be as simple as comparing unique identifiers of objects with complex ones like comparing each field. hashCode() can be as simple as returning hashCode() unique id ' String or hashCode() view.

Answer 15

Using java 8 stream api.

  List<String> list = new ArrayList<>(); list.add("one"); list.add("one"); list.add("two"); System.out.println(list); Collection<String> c = list.stream().collect(Collectors.toSet()); System.out.println(c); 

Output:

To values: [one, one, two]

After the values: [one, two]

Answer 16

 Class removeduplicates { public static void main(string args[[]) { int I; for(int =0;i'<10;I++) { system.out.println(+i); if([]I=[j]) { system.out.println(1,2,3,1,1,1,2,2,2) } } } }