ArrayList vs LinkedList

List Overview

ArrayList and LinkedList both implements List interface and their methods and results are almost identical. Vector is almost identical to ArrayList, and the difference is that Vector is synchronized. Because of this, it has an overhead than ArrayList. Normally, most Java programmers use ArrayList instead of Vector because they can synchronize explicitly by themselves.

However there are few differences between them which make one better over another depending on the requirement.

ArrayList Vs LinkedList

1) Search: ArrayList search operation is pretty fast compared to the LinkedList search operation. get(int index) in ArrayList gives the performance of O(1) while LinkedList performance is O(n).

Reason: ArrayList maintains index based system for its elements as it uses array data structure implicitly which makes it faster for searching an element in the list. On the other side LinkedList implements doubly linked list which requires the traversal through all the elements for searching an element.

2) Deletion: LinkedList remove operation gives O(1) performance while ArrayList gives variable performance: O(n) in worst case (while removing first element) and O(1) in best case (While removing last element).

Conclusion: LinkedList element deletion is faster compared to ArrayList.

Reason: LinkedList’s each element maintains two pointers (addresses) which points to the both neighbor elements in the list. Hence removal only requires change in the pointer location in the two neighbor nodes (elements) of the node which is going to be removed. While In ArrayList all the elements need to be shifted to fill out the space created by removed element.

3) Inserts Performance: LinkedList add method gives O(1) performance while ArrayList gives O(n) in worst case. Reason is same as explained for remove.

4) Memory Overhead: ArrayList maintains indexes and element data while LinkedList maintains element data and two pointers for neighbor nodes hence the memory consumption is high in LinkedList comparatively.

Performance of ArrayList vs. LinkedList

Sample code to test their performance:

ArrayList arrayList = new ArrayList();
LinkedList linkedList = new LinkedList();
 
// ArrayList add
long startTime = System.nanoTime();
for (int i = 0; i < 100000; i++) {
    arrayList.add(i);
}
long endTime = System.nanoTime();
long duration = endTime - startTime;
System.out.println("ArrayList add: " + duration);

// LinkedList add
startTime = System.nanoTime();
for (int i = 0; i < 100000; i++) {
    linkedList.add(i);
}
endTime = System.nanoTime();
duration = endTime - startTime;
System.out.println("LinkedList add: " + duration);

// ArrayList get
startTime = System.nanoTime();
for (int i = 0; i < 10000; i++) {
    arrayList.get(i);
}
endTime = System.nanoTime();
duration = endTime - startTime;
System.out.println("ArrayList get: " + duration);

// LinkedList get
startTime = System.nanoTime();
for (int i = 0; i < 10000; i++) {
    linkedList.get(i);
}
endTime = System.nanoTime();
duration = endTime - startTime;
System.out.println("LinkedList get: " + duration);

// ArrayList remove
startTime = System.nanoTime();
for (int i = 9999; i >=0; i--) {
    arrayList.remove(i);
}
endTime = System.nanoTime();
duration = endTime - startTime;
System.out.println("ArrayList remove: " + duration);

// LinkedList remove
startTime = System.nanoTime();
for (int i = 9999; i >=0; i--) {
    linkedList.remove(i);
}
endTime = System.nanoTime();
duration = endTime - startTime;
System.out.println("LinkedList remove: " + duration);

And the output is:

ArrayList  add: 13265642
LinkedList add: 9550057

ArrayList  get: 1543352
LinkedList get: 85085551

ArrayList  remove: 199961301
LinkedList remove: 85768810

 

There are few similarities between these classes which are as follows:

  • Both ArrayList and LinkedList are implementation of List interface.
  • They both maintain the elements insertion order which means while displaying ArrayList and LinkedList elements the result set would be having the same order in which the elements got inserted into the List.
  • Both these classes are non-synchronized and can be made synchronized explicitly by using Collections.synchronizedList method.
  • The iterator and listIterator returned by these classes are fail-fast (if list is structurally modified at any time after the iterator is created, in any way except through the iterator’s own remove or add methods, the iterator will throw a ConcurrentModificationException).

When to use LinkedList and when to use ArrayList?

1) As explained above the insert and remove operations give good performance (O(1)) in LinkedList compared to ArrayList(O(n)). Hence if there is a requirement of frequent addition and deletion in application then LinkedList is a best choice.

2) Search (get method) operations are fast in Arraylist (O(1)) but not in LinkedList (O(n)) so If there are less add and remove operations and more search operations requirement, ArrayList would be your best bet.

Advertisement

Battle of the Maps – HashTable vs HashMap vs TreeMap vs LinkedHashMap

Map Overview

There are 4 commonly used implementations of Map in Java SE – HashMap, TreeMap, Hashtable and LinkedHashMap.

All four classes implement the Map interface and offer mostly the same functionality. If we use one sentence to describe each implementation, it would be the following:

  1. HashMap is implemented as a hash table, and there is no ordering on keys or values.
  2. TreeMap is implemented based on red-black tree structure, and it is ordered by the key.
  3. LinkedHashMap preserves the insertion order
  4. Hashtable is synchronized, in contrast to HashMap.

This gives us the reason that HashMap should be used if it is thread-safe, since Hashtable has overhead for synchronization.

Some more differences between these data-structures in java are as follows –

  • Hashtable is the generic name for hash-based maps. In the context of the Java API, Hashtable is an obsolete class from the days of Java 1.1 before the collections framework existed. It should not be used anymore, because its API is cluttered with obsolete methods that duplicate functionality, and its methods are synchronized (which can decrease performance and is generally useless). Use ConcurrrentHashMap instead of Hashtable.
    • same as hash map
    • it does not allows null keys and null values
  • HashMap makes absolutely no guarantees about the iteration order. It can (and will) even change completely when new elements are added.
    • It has pair values(keys,values)
    • NO duplication key values
    • unordered unsorted
    • it allows one null key and more than one null values
    • Size : [32 * SIZE + 4 * CAPACITY bytes]
  • TreeMap will iterate according to the “natural ordering” of the keys according to their compareTo() method (or an externally supplied Comparator). Additionally, it implements the SortedMap interface, which contains methods that depend on this sort order.
    • Ordered and sortered version
    • based on hashing data structures
    • Size : [40 * SIZE bytes]
  • LinkedHashMap will iterate in the order in which the entries were put into the map
    • It is ordered version of map implementation
    • Based on linked list and hashing data structures
    • Size : [40 * SIZE + 4 * CAPACITY bytes]

 

Visual presentation:

Anatomize TreeMap (Red-Black Tree)

  • What is a Tree Map ?

Treemap class is like HashMap which stores key- value pairs . The major difference is that Treemap sorts the key in ascending order.

According to Java doc :

Treemap is sorted according to the natural ordering of its keys, or by a Comparator provided at map creation time, depending on which constructor is used.

This implementation provides guaranteed log(n) time cost for the containsKey, get, put and remove operations. Algorithms are adaptations of those in Cormen, Leiserson, and Rivest’s Introduction to Algorithms.

 

  • How TreeMap works in java ?

TreeMap is a Red-Black tree(https://www.cs.usfca.edu/~galles/visualization/RedBlack.html) based NavigableMap implementation.In other words , it sorts the TreeMap object keys using Red-Black tree algorithm.

Red Black algorithm is a complex algorithm . We should read the pseudo code of Red Black algorithm in order to understand the internal implementation .Red Black tree has the following properties :

1. As the name of the algorithm suggests ,color of every node in the tree is either red or black.

2. Root node must be Black in color.

3. Red node can not have a red color neighbor node.

4. All paths from root node to the null should consist the same number of black nodes .

 

  • Why and when we use TreeMap ?

We need TreeMap to get the sorted list of keys in ascending order.

 

  • What is the runtime performance of the get() method in TreeMap and HashMap ,where n represents the number of elements ?

According to TreeMap Java doc,

TreeMap implementation provides guaranteed log(n) time cost for the containsKey,get,put and remove operations.

According to HashMap Java doc :

HashMap implementation provides constant-time performance for the basic operations (get and put), assuming the hash function disperses the elements properly among the buckets.

One liner : TreeMap : log(n) HashMap : Constant time performance assuming elements disperses properly.

 

  • What is “natural ordering” in TreeMap ?

“Natural” ordering is the ordering implied by the implementation of the Comparable interface by the objects used as keys in the TreeMap. Essentially, RBTree must be able to tell which key is smaller than the other key, and there are two ways to supply that logic to the RBTree implementation:

1.Implement Comparable interface in the class(es) used as keys to TreeMap, or

2.Supply an implementation of the Comparator that would do comparing outside the key class itself.

Natural ordering is the order provided by the Comparable interface .If somebody puts the key that do not implement natural order then it will throw ClassCastException.

 

  • Why do we need TreeMap when we have sortedMap ?

sortedMap is a interface and TreeMap is the class implementing it .As we know one can not create objects of the interface . Interface tells us which methods a sortedMap implementation should provide .TreeMap is such an implementation.

 

  • Which data structure you will prefer in your code : HashMap or TreeMap ?

HashMap is faster while TreeMap is sorted .Thus we choose them according to their advantage.

If you do not want to sort the elements but just to insert and retrieve the elements then use HashMap .

But if you want to maintain the order of the elements then TreeMap should be preferred because the result is alphabetically sorted .While iterating HashMap there is no ordering of the elements ,on the other hand , TreeMap iterates in the natural key order.

 

  • What happens if the TreeMap is concurrently modified while iterating the elements ?

The iterator fails fast and quickly if structurally modified at any time after the iterator is created (in any way except through the iterator’s own remove method ). We already discussed the difference between Fail-fast and Fail safe iterators .

 

  • Which copy technique (deep or shallow ) is used by the TreeMap clone() method ?

According to docjar , clone() method returns the shallow copy of the TreeMap instance . In shallow copy object B points to object A location in memory . In other words , both object A and B are sharing the same elements .The keys and values themselves are not cloned .

 

  • Why java’s treemap does not allow an initial size ?

HashMap reallocates its internals as the new one gets inserted while TreeMap does not reallocate nodes on adding new ones. Thus , the size of the TreeMap dynamically increases if needed , without shuffling the internals. So it is meaningless to set the initial size of the TreeMap .

Anatomize LinkedHashMap

LinkedHashMap vs. HashMap

LinkedHashMap is a HashMap that also defines the iteration ordering using an additional data structure, a double linked list. By default, the iteration order is same as insertion-order. It can also be the order in which its entries were last accessed so it can be easily extended to build LRU cache.

 

Data Structure

The data structure of LinkedHashMap extends that of HashMap.

In HashMap, the data structure is based on array and linked list. An entry finds its location in the array based on its hash value. If an array element is already occupied, the new entry replaces the old entry and the old entry is linked to the new one.

In HashMap, there is no control on the iteration order. In LinkedHashMap, the iteration order is defined, either by the insertion order or access order.

LinkedHashMap differs from HashMap in that it maintains a doubly-linked list running through all of its entries. The below one is a modified example of the above data structure. It defines the iteration ordering based on the order in which keys were inserted into the map. In order to do so, the entry element is extended to keep track of the after and before element. A zero size LinkedHashMap contains just the Head element with before and after pointing to itself.

 

Below is the HashMap data structure:

 

Entry

LinkedHashMap’s Entry extends the HashMap’s Entry so it also inherits the same properties key, value, hash and the next Entry sharing the index. Other than these, it also has couple of additional properties to maintain the double-linked list, after and before entries.

 

New Entry

LinkedHashMap inherits HashMap so its internal data structure is same as that of HashMap. Apart from that it also maintains a double-linked list which is circularly linked via the sentinel node called head. Each node contains references to the previous and to the next node . A new node is always added to the end of the list. In order to do so, the last node’s and the header node’s links have to be adjusted.

  1. The new node’s next reference will point to the head.
  2. The new node’s previous reference will point to the current last node.
  3. The current last node’s next reference will point to the new node instead of head.
  4. Head’s previous reference will point to the new node.
	after = head;
	before = head.before;
	before.after = this;
	after.before = this;

 

 

Performance is likely to be just slightly below that of HashMap, due to the added expense of maintaining the linked list.

 

Access Ordered

A special LinkedHashMap(capacity, loadFactor, accessOrderBoolean) constructor is provided to create a linked hash map whose order of iteration is the order in which its entries were last accessed, from least-recently accessed to most-recently. Invoking the put or get method results in an access to the corresponding entry. If the enclosing Map is access-ordered, it moves the entry to the end of the list; otherwise, it does nothing.


    public void testLinkedHashMap() {
        LinkedHashMap lru = new LinkedHashMap(16, 0.75f, true);
        lru.put("one", null);
        lru.put("two", null);
        lru.put("three", null); 

        Iterator itr = lru.keySet().iterator();

        while (itr.hasNext()) {
            System.out.println(itr.next());
        } 
        System.out.println("** Access one, will move it to end **");

        lru.get("one"); 
        itr = lru.keySet().iterator();
        while (itr.hasNext()) {
            System.out.println(itr.next());
        }
        System.out.println("** Access two, will move it to end **");

        lru.put("two", "two");
        itr = lru.keySet().iterator();
        while (itr.hasNext()) {
            System.out.println(itr.next());
        }
    }


Result:
one
two
three
** Access one, will move it to end **

two
three
one
** Access two, will move it to end **

three
one
two

Thus in access-ordered linked hash maps, merely querying the map with get is a structural modification.

 

Iterator

In HashMap, the iterator has to traverse through each table element and the element’s own linked list, requiring time proportional to its capacity.

In LinkedHashMap, it simply has to traverse through its own double-linked list thus requires time proportional to the size of the map and not its capacity so HashMap iteration is likely to be more expensive.

 

Transfer

Re-sizing is supposed to be faster as it iterates through its double-linked list to transfer the contents into a new table array.


    void transfer(HashMap.Entry[] newTable) {
        int newCapacity = newTable.length;
        for (Entry e = header.after; e != header; e = e.after) {
            int index = indexFor(e.hash, newCapacity);
            e.next = newTable[index];
            newTable[index] = e;
        }
    }

Contains Value

containsValue() is Overridden to take advantage of the faster iterator.

 

LRU Cache

If access ordered is true, order of iteration is the order in which its entries were last accessed, from least-recently accessed to most-recently (access-order). This kind of map is well-suited to building LRU caches. The removeEldestEntry(Entry) method may be overridden to impose a policy for removing stale mappings automatically when new mappings are added to the map. For example,

    protected boolean removeEldestEntry(Map.Entry eldest) {
        return size() > maxCacheSize;
    }

 

Access Order

The main reason why one prefers LinkedHashMap over HashMap is that it can retain the order in which the elements are accessed. Below is the basic flow that a HashMap goes through to put a new entry. The blue boxes, ‘Add Entry’ and ‘Record Access’ are the ones LinkedHashMap overrides.

It overrides ‘Record Access’ to record the access order. If the user is interested in the access order, it updates its double linked list.

 

Below entry picture shows how the entry moves up the linked list. Head’s next entry will point to the latest entry accessed.

If E2 is accessed again, HashMap identifies the entry and then calls record access.

The record access is overridden in LinkedHashMap. Below is the class diagram where LinkedHashMap extends HashMap’s entry to override the recordAccess.

Double linked list vs Single Linked List


Suppose we want to remove an entry E2.

If we have a single linked list, E3’s next should point to E1 which means if want to eliminate E2, we need to know its previous entry. If it is a single linked list, we will end up traversing the entire list to search the entry previous to E2, thus the performance would in the O(n)

In case of double linked list, the previous pointer in E2 will take us to E3 in O(1).

Anatomize HashMap

HashMap – one of the favorite topics for both interviewer and interviewee!

Some of the most frequently asked questions and (logical) answers are enlisted here –

 

How HashMap works in Java

HashMap in Java works on hashing principle. It is a data structure which allows us to store object and retrieve it in constant time O(1) provided we know the key. In hashing, hash functions are used to link key and value in HashMap. Objects are stored by calling put(key, value) method of HashMap and retrieved by calling get(key) method. When we call put method, hashcode() method of key object is called so that hash function of map can find a bucket location to store value object, which is actually index of internal array, known as table. HashMap internally store mapping in form of Map.Entry object which contains both key and value object. When you want to retrieve the object, you call get() method and again pass key object. This time again key object generate same hash code (it’s mandatory for it to do so to retrieve object and that’s why HashMap keys are immutable e.g. String) and we end up at same bucket location. If there is only one object then it is returned and that’s your value object which you have stored earlier. Things get little tricky when collisions occurs. Since internal array of HashMap is of fixed size, and if you keep storing objects, at some point of time hash function will return same bucket location for two different keys, this is called collision in HashMap. In this case, a linked list is formed at that bucket location and new entry is stored as next node. If we try to retrieve object from this linked list, we need an extra check to search correct value, this is done by equals() method. Since each node contains an entry, HashMap keep comparing entry’s key object with passed key using equals() and when it return true, Map returns corresponding value. Since searching in lined list is O(n) operation, in worst case hash collision reduce a map to linked list. This issue is recently addressed in Java 8 by replacing linked list to tree to search in O(logN) time. By the way, you can easily verify how HashMap work by looking at code of HashMap.java in your Eclipse IDE, if you know how to attach source code of JDK in Eclipse.

 

How HashMap works in Java or sometime how get method work in HashMap is a very common question on Java interviews now days. Almost everybody who worked in Java knows about HashMap, where to use HashMap and difference between Hashtable and HashMap then why this interview question becomes so special? Because of the depth it offers. It has become very popular Java interview question in almost any senior or mid-senior level Java interviews. Investment banks mostly prefer to ask this question and some time even ask you to implement your own HashMap based upon your coding aptitude. Introduction of ConcurrentHashMap and other concurrent collections has also made this questions as starting point to delve into more advanced feature. let’s start the journey.

 

Have you used HashMap before or What is HashMap? Why do you use it

Almost everybody answers this with yes and then interviewee keep talking about common facts about HashMap like HashMap accept null while Hashtable doesn’t, HashMap is not synchronized, HashMap is fast and so on along with basics like its stores key and value pairs etc. This shows that person has used HashMap and quite familiar with the functionality it offers, but interview takes a sharp turn from here and next set of follow-up questions gets more detailed about fundamentals involved with HashMap in Java . Interviewer strike back with questions like –

 

Do you Know how HashMap works in Java or How does get () method of HashMap works in Java

And then you get answers like, I don’t bother its standard Java API, you better look code on Java source or Open JDK; I can find it out in Google at any time etc. But some interviewee definitely answer this and will say HashMap works on principle of hashing, we have put(key, value) and get(key) method for storing and retrieving Objects from HashMap. When we pass Key and Value object to put() method on Java HashMap, HashMap implementation calls hashCode method on Key object and applies returned hashcode into its own hashing function to find a bucket location for storing Entry object, important point to mention is that HashMap in Java stores both key and value object as Map.Entry in bucket which is essential to understand the retrieving logic. If people fails to recognize this and say it only stores Value in the bucket they will fail to explain the retrieving logic of any object stored in Java HashMap . This answer is very much acceptable and does make sense that interviewee has fair bit of knowledge on how hashing works and how HashMap works in Java. But this is just start of story and confusion increases when you put interviewee on scenarios faced by Java developers on day by day basis. Next question could be about collision detection and collision resolution in Java HashMap e.g.

 

What will happen if two different objects have same hashcode?

Now from here onwards real confusion starts, Some time candidate will say that since hashcode is equal, both objects are equal and HashMap will throw exception or not store them again etc, Then you might want to remind them about equals() and hashCode() contract that two unequal object in Java can have same hash code. Some will give up at this point and few will move ahead and say “Since hashcode is same, bucket location would be same and collision will occur in HashMap, Since HashMap use LinkedList to store object, this entry (object of Map.Entry comprise key and value ) will be stored in LinkedList. Great this answer make sense though there are many collision resolution methods available like linear probing and chaining, this is simplest and HashMap in Java does follow this. But story does not end here and interviewer asks –

 

How will you retrieve Value object if two Keys will have same hashcode?

how HashMap works internally in JavaInterviewee will say we will call get() method and then HashMap uses Key Object’s hashcode to find out bucket location and retrieves Value object but then you need to remind him that there are two Value objects are stored in same bucket , so they will say about traversal in LinkedList until we find the value object , then you ask how do you identify value object because you don’t have value object to compare ,Until they know that HashMap stores both Key and Value in LinkedList node or as Map.Entry they won’t be able to resolve this issue and will try and fail.

But those bunch of people who remember this key information will say that after finding bucket location , we will call keys.equals() method to identify correct node in LinkedList and return associated value object for that key in Java HashMap . Perfect this is the correct answer.

In many cases interviewee fails at this stage because they get confused between hashCode() and equals() or keys and values object in Java HashMap which is pretty obvious because they are dealing with the hashcode() in all previous questions and equals() come in picture only in case of retrieving value object from HashMap in Java. Some good developer point out here that using immutable, final object with proper equals() and hashcode() implementation would act as perfect Java HashMap keys and improve performance of Java HashMap by reducing collision. Immutability also allows caching there hashcode of different keys which makes overall retrieval process very fast and suggest that String and various wrapper classes e.g. Integer very good keys in Java HashMap.

 

Now if you clear this entire Java HashMap interview, You will be surprised by this very interesting question “What happens On HashMap in Java if the size of the HashMap exceeds a given threshold defined by load factor ?”. Until you know how HashMap works exactly you won’t be able to answer this question. If the size of the Map exceeds a given threshold defined by load-factor e.g. if load factor is .75 it will act to re-size the map once it filled 75%. Similar to other collection classes like ArrayList, Java HashMap re-size itself by creating a new bucket array of size twice of previous size of HashMap , and then start putting every old element into that new bucket array. This process is called rehashing because it also applies hash function to find new bucket location.

If you manage to answer this question on HashMap in Java you will be greeted by “do you see any problem with resizing of HashMap in Java” , you might not be able to pick the context and then he will try to give you hint about multiple thread accessing the Java HashMap and potentially looking for race condition on HashMap in Java.

So the answer is Yes there is potential race condition exists while resizing HashMap in Java, if two thread at the same time found that now HashMap needs resizing and they both try to resizing. on the process of resizing of HashMap in Java , the element in bucket which is stored in linked list get reversed in order during there migration to new bucket because Java HashMap doesn’t append the new element at tail instead it append new element at head to avoid tail traversing. If race condition happens then you will end up with an infinite loop. Though this point you can potentially argue that what the hell makes you think to use HashMap in multi-threaded environment to interviewer 🙂

 

Why String, Integer and other wrapper classes are considered good keys ?

String, Integer and other wrapper classes are natural candidates of HashMap key, and String is most frequently used key as well because String is immutable and final,and overrides equals and hashcode() method. Other wrapper class also shares similar property. Immutabiility is required, in order to prevent changes on fields used to calculate hashCode() because if key object return different hashCode during insertion and retrieval than it won’t be possible to get object from HashMap. Immutability is best as it offers other advantages as well like thread-safety, If you can keep your hashCode same by only making certain fields final, then you go for that as well. Since equals() and hashCode() method is used during reterival of value object from HashMap, its important that key object correctly override these methods and follow contact. If unequal object return different hashcode than chances of collision will be less which subsequently improve performance of HashMap.

 

Can we use any custom object as key in HashMap ?

This is an extension of previous questions. Ofcourse you can use any Object as key in Java HashMap provided it follows equals and hashCode contract and its hashCode should not vary once the object is inserted into Map. If custom object is Immutable than this will be already taken care because you can not change it once created.

 

HashMap Changes in JDK 1.7 and JDK 1.8.

There is some performance improvement done on HashMap and ArrayList from JDK 1.7, which reduce memory consumption. Due to this empty Map are lazily initialized and will cost you less memory. Earlier, when you create HashMap e.g. new HashMap() it automatically creates array of default length e.g. 16. After some research, Java team founds that most of this Map are temporary and never use that many elements, and only end up wasting memory. Also, From JDK 1.8 onwards HashMap has introduced an improved strategy to deal with high collision rate. Since a poor hash function e.g. which always return location of same bucket, can turn a HashMap into linked list, i.e. converting get() method to perform in O(n) instead of O(1) and someone can take advantage of this fact, Java now internally replace linked list to a binary tree once certain threshold is breached. This ensures performance or order O(log(n)) even in worst case where hash function is not distributing keys properly.

JAXB and Unmapped Properties

JAXB (JSR-222) is configuration by exception, meaning that there is a default mapping applied to domain objects. This means that sometimes you need to explicitly exclude a field/property. In this post I’ll discuss how this can be done using @XmlTransient or @XmlAccessorType(XmlAccessType.NONE) and when each option is appropriate.

@XmlTransient

package blog.unmapped; 

import java.util.List;
import javax.xml.bind.annotation.*; 

@XmlAccessorType(XmlAccessType.FIELD)
public class Customer {
    @XmlTransient
    private int id; 

    private String firstName;

    private String lastName; 

    private Address billingAddress; 

    private Address shippingAddress; 

    private List<PhoneNumber> phoneNumbers;

The resultant XML will not have the “id” field inside it.

If more than half of the fields/properties are unmapped then we stop gaining the benefit of configuration by exception since we need to do more work to exclude properties with @XmlTransient than we would have had to do to map the mapped properties.

To Resolve this problem, JAXB comes with a new flavor of annotation –

@XmlAccessorType(XmlAccessType.NONE)

package blog.unmapped;

import java.util.List;
import javax.xml.bind.annotation.*;

@XmlAccessorType(XmlAccessType.NONE)
public class Customer {
    private int id;

    @XmlElement
    private String firstName;

    @XmlElement
    private String lastName;

    private Address billingAddress;

    private Address shippingAddress;

    private List<PhoneNumber> phoneNumbers;
}

The resultant XML file will not have those elements which are not explicitly mapped using the @XmlElement annotation.

JDOM 2.X and Maven

XML and Java ? A tumultuous love story…

JDOM is a powerful Java library which aim to ease XML manipulation. It was once included in JSR 102 however the JSR is now withdrawn (more information can be found in the JDOM mailing list).

One of the downside of current JDOM version 2.0.2 is maven integration. When you want to include JDOM in your project, you add the following dependency:


<dependency>
  <groupId>org.jdom</groupId>
  <artifactId>jdom</artifactId>
  <version>2.0.2</version>
</dependency>

 

And JDOM pom.xml references those dependencies :

<dependencies>
  <dependency>
    <groupId>jaxen</groupId>
    <artifactId>jaxen</artifactId>
    <version>1.1.3</version>
    <optional>true</optional>
  </dependency>

  <dependency>
    <groupId>xerces</groupId>
    <artifactId>xercesImpl</artifactId>
    <version>2.10.0</version>
    <optional>true</optional>
  </dependency>

  <dependency>
    <groupId>xalan</groupId>
    <artifactId>xalan</artifactId>
    <version>2.7.1</version>
    <optional>true</optional>
  </dependency>
</dependencies>

 

Now if you try to perform mvn install, you will face one of the following errors :

  • The following artifacts could not be resolved: maven-plugins:maven-cobertura-plugin:plugin:1.3, maven-plugins:maven-findbugs-plugin:plugin:1.3.1, Failure to find maven-plugins:maven-cobertura-plugin:plugin:1.3 in …

This error is caused by a broken artifact on Jaxen 1.1.3. The work-around is to use the “exclusions” tags in order to avoid loading of unfound dependencies.

<dependency>
  <groupId>jaxen</groupId>
  <artifactId>jaxen</artifactId>
  <version>1.1.3</version>
  <optional>true</optional>
  <exclusions>
    <exclusion>
      <groupId>maven-plugins</groupId>
      <artifactId>maven-cobertura-plugin</artifactId>
    </exclusion>
    <exclusion>
      <groupId>maven-plugins</groupId>
      <artifactId>maven-findbugs-plugin</artifactId>
    </exclusion>
  </exclusions>
</dependency>

 

 

java.lang.ExceptionInInitializerError
    at org.jdom2.input.SAXBuilder.(SAXBuilder.java:338)
    at org.jdom2.input.SAXBuilder.(SAXBuilder.java:221)
    at [...]

Caused by: java.lang.UnsupportedOperationException: This parser does not support specification "null" version "null"

    at javax.xml.parsers.SAXParserFactory.setSchema(SAXParserFactory.java:421)
    at org.jdom2.input.sax.XMLReaders.(XMLReaders.java:122)
    at org.jdom2.input.sax.XMLReaders.(XMLReaders.java:95)
    ... 30 more

 

This error is caused by an old version of xercesImpl specified in the resolved dependencies. Once again, we will use the “exclusions” tag to avoid this error.

<dependency>
  <groupId>jaxen</groupId>
  <artifactId>jaxen</artifactId>
  <version>1.1.3</version>
  <optional>true</optional>

  <!-- Use JDK xerces version -->
  <exclusions>
    <exclusion>
      <groupId>xerces</groupId>
      <artifactId>xercesImpl</artifactId>
    </exclusion>
  </exclusions>
</dependency>

 

This should not work if your JDK is lesser than 1.5.

In summary, here is the depency set I currently use for JDOM 2.0.2 :

<dependency>
  <groupId>org.jdom</groupId>
  <artifactId>jdom</artifactId>
  <version>2.0.2</version>
</dependency>

<dependency>
  <groupId>jaxen</groupId>
  <artifactId>jaxen</artifactId>
  <version>1.1.3</version>
  <exclusions>
    <!-- Use JDK xerces version -->
    <exclusion>
      <groupId>xerces</groupId>
      <artifactId>xercesImpl</artifactId>
    </exclusion>
    <exclusion>
      <groupId>maven-plugins</groupId>
      <artifactId>maven-cobertura-plugin</artifactId>
    </exclusion>
    <exclusion>
      <groupId>maven-plugins</groupId>
      <artifactId>maven-findbugs-plugin</artifactId>
    </exclusion>
  </exclusions>
</dependency>
<dependency>
</dependency>

 

Spring Integration – Communication with legacy servers using TCP sockets

I recently faced a situation where I had to communicate with a legacy server (which only knows TCP – Sockets as communication channel). The client side application was entirely Spring (v4.0) based and I had to lookout for a standard way from the Spring arena to make this communication happen.

Read many articles over the net regarding Spring Integration module and finally came up with the following spring context, configuration and java code to establish the communication –

geocodeViaTCPCommunicationContext.xml –


<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"    
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:int="http://www.springframework.org/schema/integration"    
       xmlns:int-ip="http://www.springframework.org/schema/integration/ip"
       xmlns:context="http://www.springframework.org/schema/context"    
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                           http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
                           http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
                           http://www.springframework.org/schema/integration/ip http://www.springframework.org/schema/integration/ip/spring-integration-ip.xsd">
    <context:property-placeholder />
    <!-- Client side -->
    <int:gateway id="tcpGateway" service-interface="com.geocode.Gateway" default-request-channel="input"/>
    <int-ip:tcp-connection-factory id="client"
        type="client"
        host="${tcp.server.host}"
        port="${tcp.server.port}"
        single-use="true"
        so-timeout="100000"        
        serializer="connectionSerializeDeserialize"
        deserializer="connectionSerializeDeserialize"/>
    <int:channel id="input" />

    <int-ip:tcp-outbound-gateway id="outGateway"
        request-channel="input"
        reply-channel="clientBytes2StringChannel"
        connection-factory="client"
        request-timeout="100000"
        reply-timeout="100000"/>

    <int:object-to-string-transformer id="clientBytes2String" input-channel="clientBytes2StringChannel" />        
    <bean id="connectionSerializeDeserialize" class="org.springframework.integration.ip.tcp.serializer.ByteArrayLengthHeaderSerializer"/>
</beans>

 

GeocodeServiceImpl.java –

import java.io.StringReader;
import java.io.StringWriter;
import java.util.List;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBElement;
import javax.xml.bind.Marshaller;
import javax.xml.bind.Unmarshaller;
import javax.xml.transform.Source;
import org.jdom2.Document;
import org.jdom2.Element;
import org.jdom2.input.SAXBuilder;
import org.jdom2.transform.JDOMSource;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import geocode.Gateway;
import geocode.GeocodeRequest;
import geocode.GeocodeResponse;
import service.GeocodeService; 

@Service("geocodeService")
public class GeocodeServiceImpl implements GeocodeService {

    @Autowired
    Gateway tcpGateway;

    @Override
    public GeocodeResponse geocode(GeocodeRequest request) {
        try {
            JAXBContext jaxbContext = JAXBContext.newInstance(GeocodeRequest.class);
            Marshaller jaxbMarshaller = jaxbContext.createMarshaller(); 

            // output pretty printed
            jaxbMarshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true);
            jaxbMarshaller.marshal(request, System.out);

            StringWriter w = new StringWriter();
            jaxbMarshaller.marshal(request, w); 

            String responseString = tcpGateway.send(w.toString()); //send the marshalled request to legacy server and get response
            SAXBuilder builder = new SAXBuilder();
            Document responseDocument = builder.build(new StringReader(responseString)); //convert response into JDOM document
            Element responseEle = responseDocument.getRootElement().getChild("RESPONSE");

            if ("FAILED".equals(responseEle.getChildText("STATUS")) || responseEle.getChild("NOMATCHFOUND") != null || responseDocument.getRootElement().getChild("RESPONSE").getChild("STATUS").getText().equalsIgnoreCase("FAILED")) {
                GeocodeResponse geocodeResponse = new GeocodeResponse();
                geocodeResponse.setStatus(GeocodeResponse.UNMATCHED);
                return geocodeResponse;
            } else {
                List candidates = responseDocument.getRootElement().getChild("RESPONSE").getChildren("CANDIDATE");
                Element candidate1 = (Element) candidates.get(0);
                Source src = new JDOMSource(candidate1);
                JAXBContext jaxbResponseContext = JAXBContext.newInstance(GeocodeResponse.class); //Un-marshall the response into java object
                Unmarshaller u = jaxbResponseContext.createUnmarshaller();
                JAXBElement element = u.unmarshal(src, GeocodeResponse.class);
                GeocodeResponse geocodeResponse = (GeocodeResponse) element.getValue();
                geocodeResponse.setStatus(GeocodeResponse.MATCHED);
                return geocodeResponse;
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }
}

The pom.xml additional dependencies include –

 <dependency>
   <groupId>org.springframework.integration</groupId>
   <artifactId>spring-integration-test</artifactId>
   <version>${org.springframework-version}</version>
   <scope>compile</scope>
 </dependency>
 
 <dependency>
   <groupId>org.springframework</groupId>
   <artifactId>spring-test</artifactId>
   <version>${org.springframework-version}</version>
   <scope>test</scope>
 </dependency>
 
 <dependency>
   <groupId>org.springframework.integration</groupId>
   <artifactId>spring-integration-ip</artifactId>
   <version>${org.springframework-version}</version>
   <scope>compile</scope>
 </dependency></pre>
<pre>

Loading configuration properties from DataBase in Spring based application

Its better to store the configuration properties in a database table so that it can be managed easily for different environments (dev/QA/prod). This post explains the approach to store and retrieve the configuration properties from database table in a enterprise spring based project.

Configure the below bean definition to the spring context file

<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
 <property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE" />
 <property name="properties">
 <bean class="org.apache.commons.configuration.ConfigurationConverter" factory-method="getProperties">
 <constructor-arg>
 <bean class="org.apache.commons.configuration.DatabaseConfiguration">
 <constructor-arg>
 <ref bean="dbDataSource" />
 </constructor-arg>
 <constructor-arg value="DOMAIN_CONFIG" /> <!-- DB Table -->
 <constructor-arg value="CONFIG_NAME" /> <!-- DB Key Column -->
 <constructor-arg value="CONFIG_VALUE" /> <!-- DB Value Column -->
 </bean>
 </constructor-arg>
 </bean>
 </property>
 </bean>

The Properties table should be pre-created with the required configuration values


The datasource ref bean propertyDataSource should be configured in the context file –

<bean id="dbDataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
 <property name="jndiName" value="datasource.name" />
</bean>

Now the properties can be referred as shown below –

<int-ip:tcp-connection-factory id="client"
 type="client"
 host="${iit.server.host}"
 port="${iit.server.port}"
 single-use="true"
 so-timeout="100000"
 serializer="connectionSerializeDeserialize"
 deserializer="connectionSerializeDeserialize"/>

If the same set of properties need to be referred inside the beans as well then we need to make minor adjustments to the context declaration made above. The change will be –

<bean id="domainConfiguration" class="org.apache.commons.configuration.DatabaseConfiguration">
   <constructor-arg>
     <ref bean="dbDataSource" />
   </constructor-arg>
   <constructor-arg value="ERL_3PT_DOMAIN_CONFIG" /> <!-- DB Table -->
   <constructor-arg value="CONFIG_NAME" /> <!-- DB Key Column -->
   <constructor-arg value="CONFIG_VALUE" /> <!-- DB Value Column -->
</bean>

<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
 <property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE" />
 <property name="properties">
  <bean class="org.apache.commons.configuration.ConfigurationConverter" factory-method="getProperties">
   <constructor-arg>
     <ref bean="domainConfiguration" />
   </constructor-arg>
  </bean>
 </property>
</bean></pre>
<pre>

And a service in java class can be written as follows –

import java.util.Properties;

import org.apache.commons.configuration.ConfigurationConverter;
import org.apache.commons.configuration.DatabaseConfiguration;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.stereotype.Service;

/**
 * Service to retrieve the properties stored in the DOMAIN_CONFIG table. This service can be autowired in any other service and its getProperty method can be invoked to get the property value. 
 * The getProperty() method makes sure that all the properties from the database are loaded only once throughout the life time of the current application session.
 * @author akhandel
 *
 */
@Service("domainConfigService")
public class DomainConfig {

 @Autowired
 @Qualifier("domainConfiguration")
 DatabaseConfiguration domainConfiguration;

 private Properties properties;

 public String getProperty(String key) {
 if (properties == null) {
 properties = ConfigurationConverter.getProperties(domainConfiguration);
 }
 return properties.getProperty(key);
 }
}</pre>
<pre>

Web-Based Excel Report Using JXLS and SpringMVC

Let’s create a simple excel exporter report using SpringMVC.

Im integrating JXLS library with SpringMVC framework. I prefer using JXLS compared to JasperReport or other java-to-excel-library due to its easy templating, so i dont need to create excel formatting from java code.

 

Let’s start with pom.xml :


<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <modelVersion>4.0.0</modelVersion>
    <groupId>com.edw</groupId>
    <artifactId>JXLSSpringMVC</artifactId>
    <version>1.0</version>
    <packaging>war</packaging>
    <name>JXLSSpringMVC</name>

    <properties>
        <endorsed.dir>${project.build.directory}/endorsed</endorsed.dir>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>
    <dependencies>
        <dependency>
            <groupId>javax</groupId>
            <artifactId>javaee-web-api</artifactId>
            <version>6.0</version>
            <scope>provided</scope>
        </dependency>

     <!-- Spring 3 dependencies -->

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
            <version>3.1.1.RELEASE</version>
        </dependency>

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
            <version>3.1.1.RELEASE</version>
        </dependency>

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-webmvc</artifactId>
            <version>3.1.1.RELEASE</version>
        </dependency>

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-tx</artifactId>
            <version>3.1.1.RELEASE</version>
        </dependency>

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-orm</artifactId>
            <version>3.1.1.RELEASE</version>
        </dependency>

        <dependency>
            <groupId>cglib</groupId>
            <artifactId>cglib</artifactId>
            <version>2.2</version>
        </dependency>

        <dependency>
            <groupId>commons-codec</groupId>
            <artifactId>commons-codec</artifactId>
            <version>1.9</version>
        </dependency>

        <!-- jxls -->

        <dependency>
            <groupId>net.sf.jxls</groupId>
            <artifactId>jxls-core</artifactId>
            <version>0.9.9</version>
        </dependency>
    </dependencies> 

    <build>
        <plugins>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.3.2</version>
                <configuration>
                    <source>1.6</source>
                    <target>1.6</target>
                    <compilerArguments>
                        <endorseddirs>${endorsed.dir}</endorseddirs>
                    </compilerArguments>
                </configuration>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-war-plugin</artifactId>
                <version>2.1.1</version>
                <configuration>
                    <failOnMissingWebXml>false</failOnMissingWebXml>
                </configuration>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-dependency-plugin</artifactId>
                <version>2.1</version>
                <executions>
                    <execution>
                        <phase>validate</phase>
                        <goals>
                            <goal>copy</goal>
                        </goals>

                        <configuration>
                            <outputDirectory>${endorsed.dir}</outputDirectory>
                            <silent>true</silent>
                            <artifactItems>
                                <artifactItem>
                                    <groupId>javax</groupId>
                                    <artifactId>javaee-endorsed-api</artifactId>
                                    <version>6.0</version>
                                    <type>jar</type>
                                </artifactItem>
                            </artifactItems>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

        </plugins>
    </build>
</project>

 

web.xml file :



<?xml version="1.0" encoding="UTF-8"?>
<web-app version="3.0" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd">
    <context-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>/WEB-INF/applicationContext.xml</param-value>
    </context-param>

    <listener>
        <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
    </listener>


    <servlet>
        <servlet-name>dispatcher</servlet-name>
        <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
        <load-on-startup>2</load-on-startup>
    </servlet>


    <servlet-mapping>
        <servlet-name>dispatcher</servlet-name>
        <url-pattern>/</url-pattern>
    </servlet-mapping>


    <session-config>
        <session-timeout>
            30
        </session-timeout>
    </session-config>

</web-app>

 

2 SpringMVC configuration files –

1. applicationContext.xml :

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:tx="http://www.springframework.org/schema/tx"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
        http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.0.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
        http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-3.0.xsd
        http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.0.xsd">

    <context:annotation-config/>
    <tx:annotation-driven/>
    <context:component-scan base-package="com.edw.jxlsspringmvc"/>
</beans>

2. dispatcher-servlet.xml :

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:p="http://www.springframework.org/schema/p"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:mvc="http://www.springframework.org/schema/mvc"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
        http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-3.0.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd">

    <context:component-scan base-package="com.edw.jxlsspringmvc.controller" />
    <mvc:annotation-driven />
    <bean id="viewResolver"
          class="org.springframework.web.servlet.view.InternalResourceViewResolver"
          p:prefix="/WEB-INF/jsp/"
          p:suffix=".jsp" />
</beans>

Simple excel file as template :


Jjava controller file:

package com.edw.jxlsspringmvc.controller; 

import java.io.FileInputStream;
import java.util.HashMap;
import java.util.Map;
import javax.servlet.ServletContext;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import net.sf.jxls.transformer.XLSTransformer;
import org.apache.poi.ss.usermodel.Workbook;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;

 

@Controller
public class IndexController {

    @Autowired
    private ServletContext context; 

    @RequestMapping(value = "/", method = RequestMethod.GET)
    public String index() {
        return "index";
    } 

    @RequestMapping(value = "/export", method = RequestMethod.GET)
    public String export(HttpServletRequest request, HttpServletResponse response) {
        try {
            // set output header
            ServletOutputStream os = response.getOutputStream();
            response.setContentType("application/vnd.ms-excel");
            response.setHeader("Content-Disposition", "attachment; filename=\"myexcel.xls\"");
            String reportLocation = context.getRealPath("WEB-INF");
            Map beans = new HashMap();
            beans.put("name", "Edwin");
            beans.put("address", "Jakarta, Indonesia");

            XLSTransformer transformer = new XLSTransformer(); 
            Workbook workbook = transformer.transformXLS(new FileInputStream(reportLocation + "/myexcel.xls"), beans);
            workbook.write(os);
            os.flush();
            return null;
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }
}

 

index.jsp file :

<html>
    <head>
        <title>Download Excel</title>
    </head>
    <body>
        <a href="${pageContext.request.contextPath}/export">download excel</a>
    </body>
</html>

 

Netbeans project structure :


And this is the result on clicking on the website’s download url :