In our previous post we saw about implementing Create operation for creating a new entity and persisting to the database. In this article lets see how we can retrieve all the entities stored in the datastore. This would be the Read operation from the CRUD operations in Gaelyk

Read- CRUD Operations in Gaelyk

We would continue with the project created in the previous post, so for those who haven’t please read the previous post and then continue with this post.

The DatastoreService provides synchronous access to the datastore to perform create, read, delete, update operations on the Entity. It also provides transaction support. Gaelyk provides shortcuts for the operations supported by Google App Engine and on similar lines DatastoreService can be accessed by using datastore shorthand.One can read here about the shorthands provided by Gaelyk for DatastoreService.

And as mentioned in previous post PROJECT_HOME is the location where the Gaelyk Template Project was extracted. Lets create listBooks.groovy groovlet in PROJECT_HOME/src/main/webapp/WEB-INF/groovy folder:

def books = datastore.execute{
    select all from book
}
request.books = books
forward '/WEB-INF/pages/listBook.gtpl'

All the groovy scripts in PROJECT_HOME/src/main/webapp/WEB-INF/groovy folder are the groovlets, just like the servlets we have in Java world. One can read about the datastore.execute syntax here. We fetch all the entities named “book” and store it in books reference and set this data to the request attribute. The books reference is actually some iterator which can be traversed using the each function. In Dynamically typed languages such as Groovy you really not care what the type is, we are concerned with what methods can be invoked on that object. Its something called “Duck Typing”. Lets have a look at listBook.gtpl which renders the data retrieved from above. Create listBook.gtpl in PROJECT_HOME/src/main/webapp/WEB-INF/pages:

//listBook.gtpl
<% include '/WEB-INF/includes/header.gtpl' %>
<div class="row">
  <div class="span1"><h4>ISBN</h4></div>
  <div class="span3"><h4>Title</h4></div>
  <div class="span3"><h4>Authors</h4></div>
  <div class="span2"><h4>Publisher</h4></div>
</div>
<%
request.books.each{
  book ->
%>
  <div class="row">
    <div class="span1"><%= book.isbn %></div>
    <div class="span3"><%= book.title %></div>
    <div class="span3"><%= book.authors %></div>
    <div class="span2"><%= book.publisher %></div>
  </div>
<%
}
%>
<% include '/WEB-INF/includes/footer.gtpl' %>

As I said before we are more interested in the methods which can be invoked on the data set in the request. And I also said that the data set is some form of iterator and hence we iterate through it by providing a closure to the each method. In the closure provided we render the HTML content for each item in the iterator. This is quite familiar for people from the JSP/PHP world.

We havent yet setup the route to access the above page, lets go ahead and edit the routes.groovy file in PROJECT_HOME/src/main/webapp/WEB-INF/ and add the following line:

get "/book/all", forward: "/listBooks.groovy"

Run your application by executing gradlew gaeRun on the terminal and then access the page using: http://localhost:8080/book/all. If you dont see any data that means you havent added any. If you followed our previous post then you must be able to add new data by using the URL: http://localhost:8080/book/new.

Once the application is up and running, on visiting the browser, one would see some content similar to:

I have committed the source code into my github project here. Though the git hub project is not exclusively for this example, but it does contain the files/code necessary for these CRUD operation examples.

We have seen here and here on creating simple mashup using Gaelyk. In this series of articles lets see how we can implement CRUD operations in Gaelyk.

CRUD Operations in Gaelyk would involve:

• Create an entity
• Read an entity
• Update an entity
• Delete an entity

In this article we will look at creating an entity in the datastore.

Create of CRUD operations

Google App Engine supports both NoSQL and SQL based database. The NoSQL counterpart uses BigTable datastore. And we would be using the same for our CRUD operations.

The data in BigTable is held in Entity where each entity has some fields, associated types and values for each field. It also adds a Key to each data row in the entity.

We will look at the details of the datastore as we proceed further, but lets setup the Gaelyk project first. The easiest way is to download the template project from here and extract it to some directory. Lets call this directory as PROJECT_HOME. One can run the command gradlew gaeRun to deploy the template project into the embedded Jetty server and access it from http://localhost:8080/.

In this series of articles we would be focused on a Book entity which has the following fields: isbn, title, authors, publisher, number of pages. To create an entity and store in the database we need to create a form for data entry, let go ahead and create a newBook.gtpl in PROJECT_HOME/src/main/webapp/WEB-INF/pages folder:

//newBook.gtpl
<% include '/WEB-INF/includes/header.gtpl' %>
<form action="/book" method="POST">
  <div class="row">
    <div class="span5">
      <label for="isbn"/>ISBN</label>
      <input id="isbn" type="text"
             name="isbn"/>

      <label for="title"/>Title</label>
      <input id="title" type="text"
             name="title"/>

      <label for="authors"/>Author(s)</label>
      <input id="authors" type="text"
             name="authors"/>

      <label for="publisher"/>Publisher</label>
      <input id="publisher" type="text"
             name="publisher"/>

      <label for="numberOfPages"/>Number of Pages</label>
      <input id="numberOfPages" type="text"
             name="numberOfPages"/>
    </div>
  </div>
  <div class="row">
    <div class="span4">
    <input type="submit" value="Save">
    <input type="reset" value="Reset">
    </div>
  </div>
</form>

<% include '/WEB-INF/includes/footer.gtpl' %>

The template project uses bootstrap and therefor the styling aspects would be taken care! Now that we have a form to enter the data, we would need to setup up the route to access this page, that way we will have clean urls and not ending with .gtpl or .groovy and so on. The routes.groovy file handles all the routing information and is present in PROJECT_HOME/src/main/webapp/WEB-INF folder. Lets add the following entry to the routes.groovy file

//routes.groovy
get "/book/new", forward: "/WEB-INF/pages/newBook.gtpl"

the above code says that when there is a GET request for ‘/book/new’, forward the request to the newBook.gtpl which renders the html markup for the page. On running the application locally using gradlew gaeRun and navigating to http://localhost:8080/book/new you should be able to see your page as something like:

Wait! we havent still configured the routes for Save operations, lets go ahead of do that. Lets create a groovy script addBook.groovy in the PROJECT_HOME/src/main/webapp/WEB-INF/groovy folder where all the groovy scripts(groovlets) are to be added.

//addBook.groovy
import com.google.appengine.api.datastore.Entity
def entity = new Entity("book")
entity << params
entity.save()
request.operation = "saved"
request.savedTitle = params.title

forward '/WEB-INF/pages/success.gtpl'

params is a map which contains the data submitted by the form as key-value pairs. One can inspect the data by logging params.isbn or params.title or any of the params.fieldName data on to the console. Just in case someone wants to try use log.info params.title in the above groovy script.

entity << params
entity.save()

the above code appends the key-value pairs in the params object to the entity object we have created and then save the contents on the entity object to the datastore. After saving the entity we forward to another page- success.gtpl which is created in PROJECT_HOME/src/main/webapp/WEB-INF/pages folder.

//success.gtpl
<% include '/WEB-INF/includes/header.gtpl' %>
<div class="row">
    <p>Successfully <%=  request.operation %> the
       book <%= request.savedTitle %></p>
    <p><a href="/book/new">Add another book</a></p>
</div>

<% include '/WEB-INF/includes/footer.gtpl' %>

We are yet to setup the route information for the groovy script we created to add the books. So go back to routes.groovy and add the following to the routes.groovy

//routes.groovy
post "/book", forward: "/addBook.groovy"

which says, that any POST requests for /book should be forwarded to addBook.groovy script/groovlet. So save all the files and reload the URL http://localhost:8080/book/new and add some entries to the form and click on “Save” button and you should be able to see the “success” page.

Google App Engine provides a way to access the entities/data stored in the datastores by navigating to: http://localhost:8080/_ah/admin/datastore.

In the next article lets see how we can list all the entities we have stored in the datastore.

We have seen how to develop a ZK based application and deploy it to OpenShift. Google has also a Paas offering called Google App Engine(GAE) which supports Java, Python and Dart applications. Gaelyk is a simple Groovy based toolkit to develop and deploy application to GAE. In this article lets build a simple Gaelyk based application and deploy it to GAE.

The sample application developed would show the details of location information from where this application was accessed, information lie Name of the city, country, region, latitude, longitude and timezone would be shown.

Setting up Gaelyk

But before that we need to download and setup Gaelyk. Faster way to start with Gaelyk is to download the Template project and extract its contents. Gaelyk template project uses Gradle build system to manage all the dependencies. To draw an analogy Gradle is similar to Maven. The template project has defined all the require dependencies and also provides a Gradle wrapper gradlew using which you can run gradle tasks without having to install Gradle. Also Gradle manages downloading and configuring the Google App Engine SDK, so you need not worry about that.

$ gradlew tasks

shows the available tasks, it might download the dependencies it doesnt find in its local repository. To run the template project:

$ gradlew gaeRun

The application will be deployed in the embedded jetty container and can be accessed at http://localhost:8080/. Gaelyk builds on top of Groovlets and Groovy templates. Groovy templates and Groovlets reside in the WEB-INF/pages and WEB-INF/groovy folders respectively. Also all the GAE services can be accessed from with in these Groovlets and Groovy Templates using the shortcodes provided by Gaelyk like memcache for accessing the caching service, datastore for accessing the GAE Datastore service and so on. For more details one can refer to more detailed Gaelyk tutorials.

Location Information from IP Address

For this we make use of the API provided by the IPInfoDb. We are interested in City wise precision and the results in JSON, hence make use of the following API:

http://api.ipinfodb.com/v3/ip-city/?key=<your_api_key>&ip=<ip_address>&format=json

The above api returns result which is something like

{
  "statusCode" : "OK",
  "statusMessage" : "",
  "ipAddress" : "74.125.45.100",
  "countryCode" : "US",
  "countryName" : "UNITED STATES",
  "regionName" : "CALIFORNIA",
  "cityName" : "MOUNTAIN VIEW",
  "zipCode" : "94043",
  "latitude" : "37.3956",
  "longitude" : "-122.076",
  "timeZone" : "-08:00"
}

parsing the above JSON in Groovy we would have something like

//Setting up the JSON data.
def myIp = request.remoteAddr
def api_key="<enter the api key for IpInfoDb here>"
def ipUrl = "http://api.ipinfodb.com/v3/ip-city/?key=${api_key}&ip=${myIp}&format=json"
def jsonSlurper = new JsonSlurper()
def locationInformation = jsonSlurper.parseText(new URL(ipUrl).text)

In the above code request represents the HttpServletRequest object and to access the IP address of the client we make use of the getRemoteAddr() method of HttpServletRequest. In groovy one can invoke getPropertyName as just propertyName, hence request.remoteAddr. If you are running this locally then you might want to replace the IP address 127.0.0.1 with your actual IP address. Something like:

def myIp = request.remoteAddr
if(myIp.equals("127.0.0.1")){
    myIp = "YOUR IP ADDRESS"
}

The subsequent code makes a GET request to the API URL and parses the obtained JSON response to get the location information.

//Parsing the JSON to get the required information
locationInformation = jsonSlurper.parseText(new URL(ipUrl).text)

//We set this as parameters to the request object
request.ipAddress = locationInformation.ipAddress
request.country = locationInformation.countryName
request.region = locationInformation.regionName
request.city = locationInformation.cityName
request.latitude = locationInformation.latitude
request.longitude = locationInformation.longitude
request.timezone = locationInformation.timeZone

Gaelyk provides a flexible URL routing system where in we can map the URL to corresponding Groovlets. These routes are declared in the routes.groovy file which is present in the WEB-INF directory. Lets create a Groovlet for our Index page and call it: index.groovy and this groovlet would be responsible for intercepting the request and obtaining the required data and forwarding to the right groovy template. The template project which you would have downloaded would have a groovy template by name: index.gtpl, and the groovlet which we created would forward the request to index.gtpl. The template project has the “/” route pointing to a different handler, we would update that to point to our groovlet.

//Updating the routes.groovy
get "/", forward: "/index.groovy"

As already mentioned the groovlet: index.groovy would be responsible for obtaining the location information from the IP address and setting those values in the request, which are then displayed by the groovy template- index.gtpl

//Contents of index.groovy
import groovy.json.JsonSlurper

def myIp = request.remoteAddr
if(myIp.equals("127.0.0.1")){
  myIp = "YOUR IP ADDRESS"
}

def api_key="YOUR IPINFODB API KEY"

def ipUrl = "http://api.ipinfodb.com/v3/ip-city/?key=${api_key}&ip=${myIp}&format=json"
def jsonSlurper = new JsonSlurper()
def locationInformation = null
//Use memcache to cache the location information
if (memcache[myIp] != null){
  locationInformation = memcache[myIp]
}
else{
  locationInformation = jsonSlurper.parseText(new URL(ipUrl).text)
  memcache[myIp] = locationInformation
}
request.ipAddress = locationInformation.ipAddress
request.country = locationInformation.countryName
request.region = locationInformation.regionName
request.city = locationInformation.cityName
request.latitude = locationInformation.latitude
request.longitude = locationInformation.longitude
request.timezone = locationInformation.timeZone

//Forward to the corresponding groovy template
forward '/WEB-INF/pages/index.gtpl'

The interesting piece of information in the above code snippet is:memcache. This binding refers to the Memcache service provided by Google App Engine. We cache the location information against the IP address and any subsequent requests from the same IP would fetch from the cache instead.

The values set as part of the request parameters are then retrieved by the index.gtpl as:

<% include '/WEB-INF/includes/header.gtpl' %>
<div class="row">
  <div class="span12">
    <h3>You are accessing from</h3>
    <ul>
      <li>Your IP Address: ${request.ipAddress}</li>
      <li>Country: ${request.country}</li>
      <li>Region: ${request.region}</li>
      <li>City: ${request.city}</li>
      <li>Latitude: ${request.latitude}</li>
      <li>Longitude: ${request.longitude}</li>
      <li>Timezone: ${request.timezone}</li>
    </ul>
  </div>
</div>
<% include '/WEB-INF/includes/footer.gtpl' %>

the include is used to include other groovy templates. Groovy templates lets you embed groovy code by just like we do in JSP and PHP.

#Run the application
$ ./gradlew gaeRun

The application would be running http://localhost:8080/

One can even create a eclipse/intellij project for this by:

#For Eclipse
gradlew cleanEclipse eclipse

#For IntelliJ 
gradlew cleanIdea idea

and then load the project in your IDE.

Once we are done with the development, we can deploy to Google App Engine by using:

$ ./gradlew gaeUpload

but before that you must create an application in your Google App Engine account and update the appengine-web.xml file in src/main/webapp/WEB-INF directory by adding the appid to the XML.

To try out this sample, you need to create a new index.groovy file in src/main/webapp/WEB-INF/groovy folder. Then create index.gtpl in src/main/webapp/WEB-INF/pages. Also dont forget to update the appengine-web.xml file.

We have seen here on how to parse JSON in Java using Gson and here on how to parse JSON in Groovy. I also touched upon in brief about Groovy here. The beauty of these JVM languages is that one can invoke these APIs from Java, the only requirement is that you need to have the language jar on the classpath. In our example below we use JsonSlurper in Groovy to parse the JSON documents from Java. Sounds cool right! In one of the talks on Functional Programming, Venkat showed us how we could exploit the Software Transactional Memory feature in Clojure to create concurrent applications in Java with less pain.

With so many JVM languages coming up i.e languages which run on JVM, developers are more becoming ployglot i.e use multiple langauges in their code. Going on same lines, in the below example I show you how we could exploit the JSON API in Groovy and use it in our Java program. So all you readers out there keep in mind to add the language specific jar to your classpath. This jar tells the compiler where to find the required API information. For Groovy you can download the required Groovy Development Kit from here. Let GROOVY_HOME be the home directory of your Groovy installation. You would find a jar in the GROOVY_HOME/embeddable folder which would be named like: groovy-all-<version>.jar. This is the jar which you need to add it to the classpath for your Java code to find the right APIs.

If you are not familiar with JSON, do stop by to read about it here, if you are not familiar with Json parsing in Groovy, do read it over here.

The json document which I used is:

[
  {
    "id":"6253282",
    "name":"Twitter API",
    "screen_name":"twitterapi",
    "url":"http://dev.twitter.com",
    "followers_count":1004641,
    "friends_count":33,
    "favourites_count":24,
    "statuses_count":3277
  },
  {
    "id":"15082387",
    "name":"Sanaulla",
    "screen_name":"sanaulla",
    "url":"http://blog.sanaulla.info",
    "followers_count":241,
    "friends_count":302,
    "favourites_count":41,
    "statuses_count":1876

  }
]

The Java code which does the parsing is:

import groovy.json.JsonSlurper;

import java.io.FileNotFoundException;
import java.io.FileReader;
import java.util.List;
import java.util.Map;

public class JavaJsonParserDemo {

  public static void main(String [] args){

    String jsonSrouce = "/home/mohamed/twitterUser.json";
    JsonSlurper jsonParser = new JsonSlurper();
    try {
      List<Map> parsedData = (List<Map>)jsonParser.parse( new FileReader(jsonSrouce));
      for ( Map aObject : parsedData){
        System.out.println(aObject);
        System.out.println(aObject.get("screen_name"));
      }
    }catch (FileNotFoundException e) {
      e.printStackTrace();
    }
  }
}

If you happen to explore the source of JsonSlurper here, you can see that the parse() method invokes either parseObject() or parseArray(). In our sample JSON it would invoke parseArray(), which would return some List. Looking into parseArray() we can find that the List is actually a List of Maps. Keeping this in mind, the declaration used above is List.

Running this program the output would be:

{id=6253282, favourites_count=24, friends_count=33, name=Twitter API,
screen_name=twitterapi, statuses_count=3277, followers_count=1004641,
url=http://dev.twitter.com}
twitterapi
{id=15082387, favourites_count=41, friends_count=302, name=Sanaulla,
screen_name=sanaulla, statuses_count=1876, followers_count=241,
url=http://blog.sanaulla.info}
sanaulla

In our previous post we saw how we used Gson API to parse the Json using Java. Not spending too much time on the introduction, I would want to straight away dive into the same parsing which can be done using Groovy language. Groovy is a scripting language which runs on the JVM. It can inter-operate with the Java APIs and compiles down into byte code which can then be executed on the JVM.

In this example as well we would make use of the same json document described here.

Groovy support for Json was added in the 1.8 version when a new groovy.json package was added. So you dont have to use an external API for parsing the json documents. JsonSlurper parses the given json document into a data structure of lists and maps. Due to this conversion the name=value pairs in the json document can be accessed by object.name notation where object represents the json object and name represents the name of the attribute.

The model bean in this case is defined as:

class GroovyUser{
  def id
  def screenName
  def name
  def url
  def followersCount
  def friendsCount
  def favouritesCount
  def statusesCount

  GroovyUser(id,
        screenName,
        name,
        url,
        followersCount,
        friendsCount,
        favouritesCount,
        statusesCount){
    this.id = id
    this.screenName = screenName
    this.name = name
    this.url = url
    this.followersCount = followersCount
    this.friendsCount = friendsCount
    this.favouritesCount = favouritesCount
    this.statusesCount = statusesCount

  }

  def String toString(){
    return "Name: "+this.name+"n"+
    "ScreenName:"+this.screenName+"n"+
    "Followers: "+this.followersCount+"n"+
    "Friends: "+this.friendsCount+"n"+
    "Favourites: "+this.favouritesCount+"n"+
    "Statuses: "+this.statusesCount+"n"
  }
}

As Groovy can be used a scripting language, the following code need not be compiled instead one could directly run the code by:

$ groovy filename.groovy

You would have to download and install the required binaries and libraries for running the groovy programs. More can be found at the official site.

import groovy.json.JsonSlurper

def jsonSlurper = new JsonSlurper();

/*
Read the JSON from the file system
*/
def reader = new BufferedReader(
              new FileReader("/home/mohamed/twitterUser.json"))
def parsedData = jsonSlurper.parse(reader)
def usersList = new ArrayList<GroovyUser>()
parsedData.each {
  aUser ->
    def user = new GroovyUser(aUser.id,
    aUser.screen_name,
    aUser.name,
    aUser.url,
    aUser.followers_count,
    aUser.friends_count,
    aUser.favourites_count,
    aUser.statuses_count)
    usersList.add(user)
}

usersList.each {aUser -> println aUser}

There are few new things to note here:

• As Groovy supports closures, we are making use of the methods like “each” which is provided by the Groovy API on collections. The each method would for each element in the collection, execute the block of code provided with the each method. Something like-myCollection.each{ aData -> println aData }
• Semi colons are optional
• Type details for the variables are optional, so you can see that the variables have been declared by using “def” keyword but no type is declared.
• Look at the tremendous decrease in the number of lines of code written above. Parsing Json in Groovy is such a pleasure.

On a closing note, the difference in the lines of code in Java and Groovy might tempt few of them to learn the syntax of the language. I am sure learning groovy is pretty easy as it has a short learning curve. Over the last few years there has been a transition to polyglot programming and with the advent of new languages which run on the JVM, they can inter-operate with an existing java application with no additional overhead.

We would over the coming days try to provide a few getting started like tutorials for Groovy and other JVM languages like Scala.