We saw how to parse JSON using GSON API, in this post we would take a look at how the same GSON API can be used to create JSON from the Java objects. Please read this to get an idea of what GSON is. Diving right into the topic of this example, we would make use of the toJson method of com.google.gson.Gson class. One of the overloaded versions of toJson() takes in an source of type Object and the type information which is of class java.lang.reflect.Type.

Lets define the model which would hold the data:

class Book{

  @SerializedName("isbn")
  String isbn;

  @SerializedName("title")
  String title;

  @SerializedName("publication")
  String publication;

  @SerializedName("authors")
  String authors;

  public Book ( String isbn,
                String title,
                String publication,
                String authors)
  {
    this.isbn = isbn;
    this.title = title;
    this.publication = publication;
    this.authors = authors;
  }

}

We populate a list with some values for this model bean and this list would be converted to JSON output. Note the use of @SerializedName annotation in the model class. This indicates the name used for the attribute in the json output. We populate the list of books using:

private static List<Book> populateBooks(){

  List<Book> myBooks = new ArrayList<Book>();
  Book book = new Book("007163360X",
                       "Java Programming",
                       "McGraw-Hill Osborne",
                       "Dr (Poornachandra) Sarang");
  myBooks.add(book);

  book = new Book("1849516626",
                  "Apache Tomcat 7 Essentials",
                  "Tanuj Khare",
                  "Packt Press");
  myBooks.add(book);

  book = new Book("007179431X",
                  "Web Developer's Cookbook",
                  "Robin Nixon",
                  "McGraw-Hill Osborne");
  myBooks.add(book);

  book = new Book("9781935182962",
                  "Practical Unit Testing",
                  "Ken Rimple & Srini Penchikala",
                  "Manning");
  myBooks.add(book);

  return myBooks;
}

Once we have a list, we need to serialize it to JSON, for that we need to setup a Type information for generic List, this is not possible out of the box in Java and hence we would make use of com.google.gson.reflect.TypeToken class in the GSON API, to create the type information like:

// Get the type information for the List<Book>
Type listType = new TypeToken<List<Book>>(){}.getType();

Next up is to use this type information along with the list of books and pass it to the toJson method to obtain the JSON output.

The complete code:

import com.google.gson.Gson;
import com.google.gson.annotations.SerializedName;
import com.google.gson.reflect.TypeToken;

import java.lang.reflect.Type;
import java.util.ArrayList;
import java.util.List;

public class JsonCreatorDemo {

  /**
  * Populate the Books information into a List
  * @return
  */
  private static List<Book> populateBooks(){

    List<Book> myBooks = new ArrayList<Book>();
    Book book = new Book("007163360X",
                         "Java Programming",
                         "McGraw-Hill Osborne",
                         "Dr (Poornachandra) Sarang");
    myBooks.add(book);

    book = new Book("1849516626",
                    "Apache Tomcat 7 Essentials",
                    "Tanuj Khare",
                    "Packt Press");
    myBooks.add(book);

    book = new Book("007179431X",
                    "Web Developer's Cookbook",
                    "Robin Nixon",
                    "McGraw-Hill Osborne");
    myBooks.add(book);

    book = new Book("9781935182962",
                    "Practical Unit Testing",
                    "Ken Rimple & Srini Penchikala",
                    "Manning");
    myBooks.add(book);

    return myBooks;
  }

  public static void main(String [] args){

    List<Book> myBooks = populateBooks();

    Gson myGson = new Gson();

    // Get the type information for the List<Book>
    Type listType = new TypeToken<List<Book>>(){}.getType();

    // Serialize the List of books into JSON
    System.out.println(myGson.toJson(myBooks,listType));

  }
}

class Book{

  @SerializedName("isbn")
  String isbn;

  @SerializedName("title")
  String title;

  @SerializedName("publication")
  String publication;

  @SerializedName("authors")
  String authors;

  public Book ( String isbn,
                String title,
                String publication,
                String authors)
  {
    this.isbn = isbn;
    this.title = title;
    this.publication = publication;
    this.authors = authors;
  }

}

JSON Output:

[
  {
    "isbn":"007163360X",
    "title":"Java Programming",
    "publication":"McGraw-Hill Osborne",
    "authors":"Dr (Poornachandra) Sarang"
  },
  {
    "isbn":"1849516626",
    "title":"Apache Tomcat 7 Essentials",
    "publication":"Tanuj Khare",
    "authors":"Packt Press"
  },
  {
    "isbn":"007179431X",
    "title":"Web Developeru0027s Cookbook",
    "publication":"Robin Nixon",
    "authors":"McGraw-Hill Osborne"
  },
  {
    "isbn":"9781935182962",
    "title":"Practical Unit Testing",
    "publication":"Ken Rimple u0026 Srini Penchikala",
    "authors":"Manning"
  }
]

In our previous post, we wrote about parsing JSON using Java and GSON library. Moving further, in this article we will look at how to use classes in java.net package to invoke a RESTful Web Service and then parse the JSON response using GSON library. For this we make use of the Twitter REST API and more specifically statuses/user_timeline API. A bit about the API:

GET status/user_timeline: Returns the 20 most recent statuses posted by the authenticating user. It is also possible to request another user’s timeline by using the screen_name or user_id parameter. The other users timeline will only be visible if they are not protected, or if the authenticating user’s follow request was accepted by the protected user.

The resource URL is: http://api.twitter.com/1/statuses/user_timeline.format

The API returns lot of data, but we are interested in only the status text and the created date. Keeping that in mind, our model class will be:

class Tweet{
  public String text;
  public String created_at;
  public String toString(){
    return "Posted "+text+" at "+created_at;
  }
}

The example can be divided into:

1. Take screen name input from the user and construct the URL for the API which we call as the fetchURL.
2. Use URLConnection class and its connect() and getInputSteam() APIs to connect to the fetchURL and then obtain the input stream.
3. The input stream obtained above is used to construct the JsonReader object( from the GSON library).
4. Then parse the JSON as explained here.

Invoking the REST API:

String fetchUrl =
  "http://api.twitter.com/1/statuses/user_timeline.json?screen_name=";
//this.screenName is obtained as a user input
fetchUrl = fetchUrl+this.screenName;
URLConnection urlConnection =  new URL(fetchUrl).openConnection();
urlConnection.connect();

One can see that to invoke a REST web service, we need not have to use any other APIs then then ones provided in the java.net package. There are a lot of libraries available for invoking and creating REST APIs. But in this example we stick with java.net package. This simplicity is due to the fact that REST APIs are based on top of HTTP related protocols and uses GET, POST, PUT, DELETE and other methods supported by HTTP.

Parsing the JSON response:

JsonReader reader = new JsonReader(
      new InputStreamReader(urlConnection.getInputStream()));
JsonParser parser = new JsonParser();
JsonElement rootElement = parser.parse(reader);
JsonArray tweetsJson = rootElement.getAsJsonArray();

Populate the required model instances and add them to the List of tweets (or List<Tweet>).

List timeline = new ArrayList();
Gson myGson = new Gson();
for ( JsonElement tweetElement : tweetsJson){
  Tweet myTweet = myGson.fromJson(tweetElement, Tweet.class);
  timeline.add(myTweet);
}

Read here for details on parsing the JSON using Java and GSON API.

Here is the complete code:

import com.google.gson.Gson;
import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonParser;
import com.google.gson.stream.JsonReader;

import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;
import java.util.ArrayList;
import java.util.List;
import java.util.Scanner;

public class RestfulDemo {
  public static void main(String [] args) throws IOException{
    Scanner commandlineReader = new Scanner(System.in);
    System.out.println("Enter the screen name");
    String screenName = commandlineReader.nextLine();
    TwitterTimeline myTimeline = new TwitterTimeline(screenName);
    myTimeline.fetchTimeline();
    for ( Tweet myTweet : myTimeline.timeline){
      System.out.println(myTweet);
    }
  }
}

class TwitterTimeline {
  public String screenName;
  public List timeline;
  private String fetchUrl =
  "http://api.twitter.com/1/statuses/user_timeline.json?screen_name=";
  TwitterTimeline( String screenName) {
    this.screenName = screenName;
  }

  public void fetchTimeline() throws IOException {
    fetchUrl = fetchUrl+this.screenName;
    URLConnection urlConnection =  new URL(fetchUrl).openConnection();
    urlConnection.connect();
    JsonReader reader = new JsonReader(
         new InputStreamReader(urlConnection.getInputStream()));
    JsonParser parser = new JsonParser();
    JsonElement rootElement = parser.parse(reader);
    JsonArray tweetsJson = rootElement.getAsJsonArray();
    timeline = new ArrayList();
    Gson myGson = new Gson();
    for ( JsonElement tweetElement : tweetsJson){
      Tweet myTweet = myGson.fromJson(tweetElement, Tweet.class);
      timeline.add(myTweet);
     }
  }
}
/*
 * Model class for a Tweet
 */
class Tweet {
  public String text;
  public String created_at;
  public String toString(){
    return "Posted "+text+" at "+created_at;
  }
}

Note: Make sure you have GSON library in the classpath.

Also if you have any issues while trying out this example, do drop a comment here and we will get back to you at the earliest.

In our previous post we saw how we used Gson API to parse the Json using Java. Not spending too much time on the introduction, I would want to straight away dive into the same parsing which can be done using Groovy language. Groovy is a scripting language which runs on the JVM. It can inter-operate with the Java APIs and compiles down into byte code which can then be executed on the JVM.

In this example as well we would make use of the same json document described here.

Groovy support for Json was added in the 1.8 version when a new groovy.json package was added. So you dont have to use an external API for parsing the json documents. JsonSlurper parses the given json document into a data structure of lists and maps. Due to this conversion the name=value pairs in the json document can be accessed by object.name notation where object represents the json object and name represents the name of the attribute.

The model bean in this case is defined as:

class GroovyUser{
  def id
  def screenName
  def name
  def url
  def followersCount
  def friendsCount
  def favouritesCount
  def statusesCount

  GroovyUser(id,
        screenName,
        name,
        url,
        followersCount,
        friendsCount,
        favouritesCount,
        statusesCount){
    this.id = id
    this.screenName = screenName
    this.name = name
    this.url = url
    this.followersCount = followersCount
    this.friendsCount = friendsCount
    this.favouritesCount = favouritesCount
    this.statusesCount = statusesCount

  }

  def String toString(){
    return "Name: "+this.name+"n"+
    "ScreenName:"+this.screenName+"n"+
    "Followers: "+this.followersCount+"n"+
    "Friends: "+this.friendsCount+"n"+
    "Favourites: "+this.favouritesCount+"n"+
    "Statuses: "+this.statusesCount+"n"
  }
}

As Groovy can be used a scripting language, the following code need not be compiled instead one could directly run the code by:

$ groovy filename.groovy

You would have to download and install the required binaries and libraries for running the groovy programs. More can be found at the official site.

import groovy.json.JsonSlurper

def jsonSlurper = new JsonSlurper();

/*
Read the JSON from the file system
*/
def reader = new BufferedReader(
              new FileReader("/home/mohamed/twitterUser.json"))
def parsedData = jsonSlurper.parse(reader)
def usersList = new ArrayList<GroovyUser>()
parsedData.each {
  aUser ->
    def user = new GroovyUser(aUser.id,
    aUser.screen_name,
    aUser.name,
    aUser.url,
    aUser.followers_count,
    aUser.friends_count,
    aUser.favourites_count,
    aUser.statuses_count)
    usersList.add(user)
}

usersList.each {aUser -> println aUser}

There are few new things to note here:

• As Groovy supports closures, we are making use of the methods like “each” which is provided by the Groovy API on collections. The each method would for each element in the collection, execute the block of code provided with the each method. Something like-myCollection.each{ aData -> println aData }
• Semi colons are optional
• Type details for the variables are optional, so you can see that the variables have been declared by using “def” keyword but no type is declared.
• Look at the tremendous decrease in the number of lines of code written above. Parsing Json in Groovy is such a pleasure.

On a closing note, the difference in the lines of code in Java and Groovy might tempt few of them to learn the syntax of the language. I am sure learning groovy is pretty easy as it has a short learning curve. Over the last few years there has been a transition to polyglot programming and with the advent of new languages which run on the JVM, they can inter-operate with an existing java application with no additional overhead.

We would over the coming days try to provide a few getting started like tutorials for Groovy and other JVM languages like Scala.

In my previous post here, I gave a brief overview of JavaScript Object Notation (JSON). In this post I would go a bit further and show how one can parse the JSON documents into Java objects using the GSON library. The JSON data for this tutorial was obtained from: “https://api.twitter.com/1/users/show.json?screen_name=” where screen_name was TwitterAPI and sanaulla. So I condensed the data into the following:

[
  {
    "id":"6253282",
    "name":"Twitter API",
    "screen_name":"twitterapi",
    "url":"http://dev.twitter.com",
    "followers_count":1004641,
    "friends_count":33,
    "favourites_count":24,
    "statuses_count":3277
  },
  {
    "id":"15082387",
    "name":"Sanaulla",
    "screen_name":"sanaulla",
    "url":"http://blog.sanaulla.info",
    "followers_count":241,
    "friends_count":302,
    "favourites_count":41,
    "statuses_count":1876
  }
]

Let me call this file as twitterUser.json. It is an array of user profile data where each user profile contains: name, id, screen_name, url, followers_count, friends_count, favourites_count and statuses_count. In order to run the following code, you need to download the GSON API jar from here and add it to your classpath. Moving further, we need to create a Model class which would hold the data obtained from JSON document.

/**
* Model class for storing the selected few attributes
* for a Twitter User profile.
*/
class TwitterUser{
  private String id;
  private String name;
  private String screen_name;
  private String url;
  private int friends_count;
  private int followers_count;
  private int favourites_count;
  private int statuses_count;
  public TwitterUser(){
  }
  @Override
  public String toString(){
    return "Name: "+this.name+"n" +
    "Screen Name: "+this.screen_name+"n" +
    "Number of Friends: "+ this.friends_count + "n" +
    "Number of Followers: "+ this.followers_count +"n" +
    "Number of Status updates: "+this.statuses_count +"n" +
    "Number of favorites: "+this.favourites_count +"n";
  }
}

Before putting forth the code, let me brief about the classes where are used for parsing the Json:

• com.google.gson.Gson:
  This is the main class for using Gson. Gson is typically used by first constructing a Gson instance and then invoking toJson(Object) or fromJson(String, Class) methods on it.
• com.google.gson.JsonParser:
  A parser to parse Json into a parse tree of JsonElements
• com.google.gson.stream.JsonReader:
  Reads a JSON encoded value as a stream of tokens. This stream includes both literal values (strings, numbers, booleans, and nulls) as well as the begin and end delimiters of objects and arrays.
• com.google.gson.JsonArray:
  A class representing an array type in Json. An array is a list of JsonElements each of which can be of a different type.
• com.google.gson.JsonElement:
  A class representing an element of Json. It could either be a JsonObject, a JsonArray, a JsonPrimitive or a JsonNull.

Reading the JSON document using JsonReader, assuming the document is present in the file systems:

JsonReader reader = new JsonReader(
        new InputStreamReader(new FileInputStream(this.jsonSource)));

JsonReader has a constructor which takes in an instance of a Reader, in our case we use the InputStreamReader and pipe it to a FileInputStream.

Once we are able to read the json data, we go ahead and use JsonParser to parse the contents and obtain the array of objects which the json document stores. There is an another way to do this with out using the JsonParser explicitly, but lets stick to this as its more clearer.

JsonParser jsonParser = new JsonParser();
JsonArray userarray= jsonParser.parse(jsonReader).getAsJsonArray();

parse(JsonReader) method of the JsonParser, parses the given Json and creates a parse tree of JsonElements. And the method getAsJsonArray() returns the parsed Json as an array of json objects. In our document we are sure that the document is an array of json objects (recall, the [ ] stands for representing the arrays in json). Suppose you were not sure if those are arrays, then you can use isJsonArray() method of the JsonElement class.

Next up, is to traverse through the array and populate the data from the json object in to our model bean TwitterUser which we had defined at the beginning.

List twitterUsers = new ArrayList<>();
for ( JsonElement aUser : userArray ) {
TwitterUser aTwitterUser = myGson.fromJson(aUser, TwitterUser.class);
twitterUsers.add(aTwitterUser);
}

For each element in the JsonArray we use Gson.fromJson() to populate the data in the json object into the type defined by TwitterUser.class. How it works behind the scene is:

• It creates a new instance of TwitterUser, by invoking the default constructor.
• Sets the values of the attributes in the TwitterUser class with those obtained from the json array element. If you closely look at the names of the attributes in the TwitterUser class and the ones in the Json document- they are exactly similar- something like- screen_name, friends_count, followerr_count and so on.

The fromJson method in Gson has been overloaded to support multiple types of parameters, and from that we are using the fromJson(JsonElement json, Class<T> classOfT) version.

Now just to confirm we have parsed the correct data, iterate through the user list:

for ( TwitterUser tUser : twitterUsers)
{
  System.out.println(tUser);
}

The complete source for the code is given below:

import com.google.gson.Gson;
import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonParser;
import com.google.gson.stream.JsonReader;

import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.List;

public class JsonParserDemo {

  private String jsonSource;
  private boolean sourceFromFile;
  public JsonParserDemo(String jsonSource, boolean sourceFromFile){
    this.jsonSource = jsonSource;
    this.sourceFromFile = sourceFromFile;
  }
  public static void main(String[] args){
    JsonParserDemo jsonParserDemo =
     new JsonParserDemo("/home/mohamed/twitterUser.json", true);
    try(JsonReader jsonReader = jsonParserDemo.getJsonReader()){
      Gson myGson = new Gson();
      JsonParser jsonParser = new JsonParser();
      JsonArray userArray =  jsonParser.parse(jsonReader).getAsJsonArray();
      List twitterUsers = new ArrayList<>();
      for ( JsonElement aUser : userArray ){
        TwitterUser aTwitterUser = myGson.fromJson(aUser, TwitterUser.class);
        twitterUsers.add(aTwitterUser);
      }
      for ( TwitterUser tUser : twitterUsers){
        System.out.println(tUser);
      }
    } catch (FileNotFoundException e) {
      e.printStackTrace();
    } catch (IOException e) {
      e.printStackTrace();
    }
  }

  /**
  * Obtain the JsonReader for the given source details.
  * @return the JsonReader instance
  * @throws FileNotFoundException
  */
  private JsonReader getJsonReader () throws FileNotFoundException{
    JsonReader reader = null;
    if (sourceFromFile){
      reader = new JsonReader(
        new InputStreamReader(new FileInputStream(this.jsonSource)));
    }
    return reader;
  }
}

/**
* Model class for storing the selected few attributes
* for a Twitter User profile.
*/
class TwitterUser{
  private String id;
  private String name;
  private String screen_name;
  private String url;
  private int friends_count;
  private int followers_count;
  private int favourites_count;
  private int statuses_count;

  public TwitterUser(){
  }

  @Override
  public String toString(){
    return "Name: "+this.name+"n" +
      "Screen Name: "+this.screen_name+"n" +
      "Number of Friends: "+ this.friends_count + "n" +
      "Number of Followers: "+ this.followers_count +"n" +
      "Number of Status updates: "+this.statuses_count +"n" +
      "Number of favorites: "+this.favourites_count +"n";
  }
}

A quick JSON search on Google would yield plenty of results and more specifically the JSON homepage. None the less we wanted to share something brief and something good about JSON. Lot of you are familiar with XML and XML had been the defacto data-transfer language in the area of RESTful webservices, but off late JSON has been gaining popularity for reasons which I would explain a bit further. Before going further into XML versus JSON, let me give a brief overview of the structure of JSON documents. And for those not aware of what JSON stands for, its JavaScript Object Notation. I know the use of “JavaScript” term has more relevance and is not like the use of “Java” in the term “JavaScript”.

A JSON structure consists of name-value pairs where a collection of such name-value pairs is called an Object, and each value could be an object themselves or be some primitives like Strings, Numbers and so on. Also values can be arrays of objects or primitive values. Sounds confusing right, let me explain with an example, consider a JSON document retrieved from http://api.twitter.com/1/trends/available.json which shows the available twitter trends and their location information. Let me get a small excerpt from that document:

[
 {
   "parentid": 23424900,
   "name": "Mexico City",
   "url": "http://where.yahooapis.com/v1/place/116545",
   "country": "Mexico",
   "countryCode": "MX",
   "placeType": {
     "name": "Town",
     "code": 7
   },
   "woeid": 116545
 },
 {
   "name": "Jackson",
   "url": "http://where.yahooapis.com/v1/place/2428184",
   "country": "United States",
   "countryCode": "US",
   "placeType": {
     "name": "Town",
     "code": 7
   },
   "parentid": 23424977,
   "woeid": 2428184
 }
]

In the above example, the notation [ ] is used to represent the Arrays and anything within { } is considered to be an object literal/ or an object. We can observe that we have an array of two objects where each object represents some twitter trend details. Digging further we can observe that objects have a “name”:value pattern, this is somewhat closer to our Map representations. The “placeType” attribute has another object as its value, similarly attributes like “code”, “woeid” have Numbers as their values.

Just in case you want to learn more about this structure/syntax, you can visit JSON.org for a beautiful diagrammatic explanation.

Origin and Use of JSON: 

Douglas Crockford of JavaScript Good Parts fame was the earliest promoters of JSON notation and it was named so because  it mimics the object notation used in JavaScript. It doesn’t require JavaScript to use but its fairly easy to create JavaScript objects using the JSON data. There are lot of APIs in almost all languages which provide support for parsing and creating JSON documents and languages like Groovy provide such support within their standard APIs. NoSQL databases like CouchDB, MongoDB extensively use JSON for representing the data.

Lot of RESTful Webservice APIs out there support both XML and JSON data formats. And with JavaScript frameworks like Backbone.js it becomes really easy to consume the JSON documents provided by these RESTful APIs and create model objects. An advantage of JSON over XML is that, its really less verbose than XML. And also the JSON representation is more closer to the Object/Key-value kind of notation which lot of us Java programmers are familiar with.

In the coming posts, we would be writing on how one can parse JSON in Java and other JVM languages and also how to use it in parsing the data from a RESTful API. So before that, we thought of giving a brief overview of JSON.

Any of you have more to share about JSON, any good/bad experiences of using JSON, do leave them here as comments, we would be more than happy to update the post to add your comments.