Tuesday, July 10, 2012

JAX-RS server: beyond hello world

Warning: This article is for the legacy JAX-RS 1.x API. Consider using the more up to date JAX-RS 2.x instead. This is not a recommended article to set that up. Take a look at the more recent article Maven JavaEE7 app in Netbeans 8 that demonstrates getting a JAX-RS 2.x based service going.


In my previous article on JAX-RS I dealt with creating a client that consumes a RESTful webservice. In this article I'm going to turn it around and use JAX-RS to expose a webservice. In a way that I myself can easily consume it from a Ruby on Rails client (proper test: make it cross technology boundaries). This service is going to expose some data as JSON, although you could take the next step and expose both JSON and XML.

In the client article I posed a generic interface which allowed a client to search for projects by name. The service would have to produce the data in this format:
[{  
      "active":true,  
      "id":1,  
      "name":"Razorblade supershave mark 2",  
      "company":{  
                 "id":1,  
                 "name":"Razorblade Ent.",  
                 "employees":[{"id":1,"name":"harry"},{"id":2,"name":"sally"}]  
                }  
     },  
     {  
       "active":true,  
       "id":9,  
       "name":"fortune cookie dispenser",  
       "company":{  
                  "id":2,  
                  "name":"Unfortunate Inc.",  
                  "employees":[{"id":5,"name":"gordon f."},{"id":7,"name":"sam stone"}]  
                 }  
     },  
     { "active":true,  
       "id":15,  
       "name":"fortune cookie baking device",  
       "company":{  
                  "id":2,  
                  "name":"Unfortunate Inc.",  
                  "employees":[{"id":5,"name":"gordon f."},{"id":7,"name":"sam stone"}]  
                 }  
     }]  

In this article I'll create a service that produces this. Since its a webservice we'll have no choice but to expose it through a webserver; I'll assume you are going to be deploying this on Tomcat 7 but if you are using a JEE6 capable application server (which has JAX-RS built in) I also provide the slightly alternative setup steps. Also as I explain in the client article there are many JAX-RS implementations, for this article too I'll be using the reference implementation called Jersey for Tomcat; if you use a JEE container it will already provide its own implementation (example: JBoss 7+ comes with RestEasy). If you have no experience with Java web development or deploying an application to something like Tomcat this article is a little out of your league at this point I'm afraid. My JSF 2 and JPA 2 on Tomcat 7 article goes into great detail and might provide you the prerequisite knowledge to be able to do what his article will assume you know how to do. At the very minimum read up about the HTTP protocol as that is basically what makes and drives RESTful webservices.

Remarkably exposing a service is less involved than consuming one; in this article I'll be dealing with:
  • Maven setup / dependencies (separate steps for Tomcat and JEE containers such as JBoss)
  • how to configure Jersey/JAX-RS
  • how to expose a service to GET data
  • how to expose a service to submit (POST) data


As it turns out there are quite a few variations on how to actually expose a service; I'm picking one that to me is the more logical, natural and above all readable way to do it that should cover most needs. In the future I'll add steps to the article to setup some proper security through Tomcat 7.

Tomcat 7 setup

This article will show two ways to setup: for Tomcat and for a JEE server; the latter is far less work. I'll also add JBoss 7.1 specific information, me being a JBoss enthusiast.

Maven
To expose a Jersey based JAX-RS service we need the following Maven dependencies which are all in Maven central. Note that if you have followed the client article also, there is going to be some duplication here. Note also that this is a slightly older article so it targets Jersey 1.x, not the newer Jersey 2.x.
  <dependency>
    <groupId>com.sun.jersey</groupId>
    <artifactId>jersey-server</artifactId>
    <version>1.18.1</version>
  </dependency>
  <dependency>
    <groupId>javax.ws.rs</groupId>
    <artifactId>jsr311-api</artifactId>
    <version>1.1.1</version>
  </dependency>
  <dependency>
    <groupId>com.sun.jersey</groupId>
    <artifactId>jersey-servlet</artifactId>
    <version>1.18.1</version>
  </dependency>  
  <dependency>
    <groupId>com.sun.jersey</groupId>
    <artifactId>jersey-json</artifactId>
    <version>1.18.1</version>
  </dependency>

That jersey-servlet dependency seems to be a bit of a secret! Apparently it used to be part of the jersey-server dependency, but has been separated at one point in time.

If you deploy to a JEE6+ compliant application server you would mark the above dependencies as provided, since the application server itself does in fact provide a JAX-RS implementation. You will want to check which one because you will want to make the maven dependencies match up.

If you're not using Maven, the following dependencies are actually going to end up in the webapp (most of them transitive dependencies):
  • jersey-core 1.18.1
  • asm 3.1
  • jersey-server 1.18.1
  • jersey-servlet 1.18.1
  • jersey-json 1.18.1
  • jettison 1.1
  • activation 1.1
  • stax-api 1.0.2
  • jaxb-api 2.2.2
  • jaxb-impl 2.2.3-1
  • jackson-core-asl 1.9.2
  • jackson-jaxrs 1.9.2
  • jackson-mapper 1.9.2
  • jackson-xc 1.9.2
So compared to the jersey-client article, the asm, jersey-server and jersey-servlet dependencies have been added.

Web
The client is install, fire, go and can even work on the commandline. The webservice part of it requires additional setup however and to my amazement there are multiple ways to actually do that. I present here the one I found to most understandable version. Jersey exposes its services as a (clever) servlet, which we need to plug in the web.xml as follows:
  <servlet>
    <servlet-name>Jersey</servlet-name>
    <servlet-class>com.sun.jersey.spi.container.servlet.ServletContainer</servlet-class>
    <init-param>
      <param-name>com.sun.jersey.config.property.packages</param-name>
      <param-value>jaxrstest.server.service;org.codehaus.jackson.jaxrs</param-value>
    </init-param>
    <init-param>
      <param-name>com.sun.jersey.api.json.POJOMappingFeature</param-name>
      <param-value>true</param-value>
    </init-param>
    <load-on-startup>1</load-on-startup>
    
  </servlet>
  
  <servlet-mapping>
    <servlet-name>Jersey</servlet-name>
    <url-pattern>/REST/*</url-pattern>
  </servlet-mapping>  

Note the servlet-class; without the jersey-servlet dependency that class would be missing. This piece of configuration data contains two application specific elements you will want to know about.

First of all there is the config.property.packages init-param. This property tells the Jersey servlet in which packages we'll be putting our restful resources (or RESTful root resources as the JEE 6 tutorial likes to call it). You can put more packages than the one by separating them with a semi-colon like this:
<param-value>jaxrstest.server.service;package.some.other</param-value>

Be sure to put a package (or packages) of your own choosing here. In the above example I've also added a package that is part of the Jackson JSON mapping library; by doing so Jersey will find the Jackson mapper classes and use them. The Jackson POJO mapping features are better (read: produces more compatible JSON layouts) than what you get from Jersey itself.

Secondly there is the url-pattern that the servlet is mapped too; you should pick a pattern that does not clash with the rest of the web application. The pattern you put here is going to be used by Jersey to build up the actual URL under which services are going to be exposed.

Also note the POJOMappingFeature init parameter - this basically tells Jersey that we'll be exposing data as JSON (and thus the jersey-json dependency is needed). If you leave this parameter out you'll find that things still work... but generally not as expected.

JEE setup


Maven
Since JEE6 and up already provides JAX-RS out of the box, we don't need much; only the API to compile against.
  <dependency>
    <groupId>javax.ws.rs</groupId>
    <artifactId>jsr311-api</artifactId>
    <version>1.1.1</version>
    <scope>provided</scope>
  </dependency>

Web
No configuration is needed. We only need to add a class to our application which indicates we want to expose a JAX-RS service, and where:
@ApplicationPath("/REST")
public class JaxrsApplication extends Application {
}

This is enough for a compliant JEE6/JEE7 container to make the magic happen. JAX-RS services will be exposed under the url WEB_URL/REST, so for example http://localhost:8080/jaxrstest/REST. Put whatever application path you like.

JBoss 7.x setup

If like me you're not rich, you use the community version of JBoss and you'll be stuck with outdated libraries. Luckily you can easily upgrade the built-in RestEasy module - and you should do that because the default version has some issues, such as not being able to use adapters. To upgrade RestEasy, go to the download page to get the latest version. Then:
  • unpack the file resteasy-jboss-modules-VERSION.Final.zip from it
  • unpack that resteasy-jboss-modules-VERSION.Final.zip file into your JBoss 7.1 modules subdirectory
  • If you do it correctly, you'll be asked to overwrite files: yes to all
Done! Note: If you are using JBoss 7.2 / EAP 6.1, you need to copy the unpacked files to the modules/system/layers/base subdirectory. See my JBoss 7 getting started article to learn how to build JBoss 7.2 yourself.

Creating a model

Since I'm aiming to produce exactly the same JSON data that the client article is set to consume, I'll re-use the model classes verbatim. Again, getters and setters left out to save space.
@XmlRootElement
public class Project {

  private boolean active;
  private long id;
  private String name;

  private Company company;

  public Project(){
  }
  
  public Project(long id, String name, boolean active, long companyId, String companyName, Employee[] emps){
    this.id = id;
    this.name = name;
    this.active = active;
    this.company = new Company(companyId, companyName, emps);
  }

  // getters and setters here
}


@XmlRootElement
public class Company {
  private long id;
  private String name;
  private Employee[] employees;

  public Company(){
  }

  public Company(long id, String name, Employee[] emps){
    this.id = id;
    this.name = name;
    this.employees = emps;
  }

  // getters and setters here
}

@XmlRootElement
public Employee {
  private long id;
  private String name;

  public Employee(){
  }

  public Employee(long id, String name){
    this.id = id;
    this.name = name;
  }
  // getters and setters here
}

Only a few things to note:
  • the XmlRootElement annotation; that is a JAXB annotation that basically marks the pojo class as a complex type to be used in mapping the data; either JSON or XML.
  • the properties of the beans match the labels as you'll find them in the JSON data
  • array of employees in the JSON data, array of employees in the Java class. You can however replace the array type with a List<Employee> type if you so please, it magically works.
Convention over configuration: you're best off just using the exact same property field names as you find them in the JSON data, or else you'll have no choice but to add extra annotations to provide mapping names.

Creating a service class

Now that we have our model, its time to create a class which is going to serve as the "project" service; the service through which we are going to expose the RESTful functions that operate on our projects; the project search to be more precise. But we'll likely want to define more functions that operate on projects so from the many different ways the API can be used to expose a service, here is again the one I find the most logical. Put this class in the package you configured in that servlet init param (in my case the jaxrstest.server.service package)!
@Path("/project")
public class ProjectResource {
  // things and stuff here
}

Lets take it one step at a time. This minor piece of code maps our service to the /project path. At this point the question arises: but what is going to be the full HTTP URL? Well assuming that:
  • We're running the webserver on the localhost on port 8080
  • Our application's web context is 'jaxrstest'
The full URL of our project service is going to be:

http://localhost:8080/jaxrstest/REST/project/

If we define that in tokens:

HOST/CONTEXT/JERSEY-URL-PATTERN/PATH

But we're not there yet - we have yet to expose our project search function. What I want to do is match the pattern as I documented it in the client article, which was along the lines of:

/REST/project/search/:NAME

Where :NAME is to be replaced with a (partial) project name to search for. Given that requirement we can lay down the service declaration as follows:
  @GET @Path("/search/{name}")
  @Produces("application/json")  
  public Project[] search(@PathParam("name") String name) {
   ...
  }

Whoa, Four annotations for one service call. Of course I picked a rather involved example, so let's examine it.

First of all the @GET annotation should not leave anything up to the imagination; we expose this service as a GET request, which should be used to fetch information. The @Path annotation binds our search function to a very specific path under /project: namely /search followed by something that should be mapped to a variable parameter; the {name} part is what tells JAX-RS that it is in fact variable. The @PathParam annotation is then used to in fact bind the variable in the URL to the parameter of the method.

And then there is the @Produces annotation which again does not leave anything up to the imagination; it defines the content-type that the method will be producing. In this case we'll be exposing the data as JSON.

Lets define an actual body for the service call. Generally at this point you would of course fire off a query to a database and actually return a filtered list of results through for example JPA; in our case lets keep it simple and just create some objects:
    Project[] ret = new Project[3];
    Employee[] emps1 = {new Employee(1, "Donald"), new Employee(2,"Hewey")};
    Employee[] emps2 = {new Employee(3, "Scrooge"), new Employee(4,"321-123")};

    ret[0] = new Project(1, name + " 1", true, 1, "Super duper company", emps1);
    ret[1] = new Project(2, name + " 2", true, 1, "Super duper company", emps1);
    ret[2] = new Project(3, name + " 3", true, 2, "Super duper company 2", emps2);
    
    return ret;
This will just echo back whatever search query you pass in the project names. A useful way to mock some test data and still see that it does "something".

At this point you're ready to deploy your webapp, fire up Tomcat and invoke the URL: http://localhost:8080/jaxrstest/REST/project/search/test. During startup you can already spy some interesting logs that indicate that things are going well for us.
9-jul-2012 17:05:13 com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
  jaxrstest.server.service
9-jul-2012 17:05:13 com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
  class jaxrstest.server.service.ProjectResource

Be sure to alter that URL to your personal environment using the description I gave you earlier. If everything is correct, you'll be greeted with the JSON data as posted at the top of this article, only as one long line. If not: did you by any chance forget to set that POJOMappingFeature init parameter?

JResponse - our list friend

To receive an array and use it is easy; to produce an array (in a statically typed language like Java) is cumbersome. It would be nicer if we could just return a generic list of projects in stead of an array. This is one of those things that has vastly improved during the time it took for JAX-RS to mature into the API it is today. To make our life incredibly easy we can use the JResponse class (as apposed to the old Response class) which preserves the generic type information for us.

Translating the code is as simple as this:
  @GET @Path("/search/{name}")
  @Produces("application/json")  
  public JResponse<List<Project>> search(@PathParam("name") String name) {
    
    // normally this would do a database search
    List<Project> ret = new ArrayList<Project>();

    Employee[] emps1 = {new Employee(1, "Donald"), new Employee(2,"Hewey")};
    Employee[] emps2 = {new Employee(3, "Scrooge"), new Employee(4,"321-123")};

    ret.add(new Project(1, name + " 1", true, 1, "Super duper company", emps1));
    ret.add(new Project(2, name + " 2", true, 1, "Super duper company", emps1));
    ret.add(new Project(3, name + " 3", true, 2, "Super duper company 2", emps2));
    
    return JResponse.ok(ret).build();
  }

This should produce exactly the same JSON output as the array example. A general use case would be: fetch list of Project instances from the database using JPA, expose it through JAX-RS service. Easy and only a few lines of code involved.

What about a client?

The client article should be able to help you there, but I'm a firm believer of crossing technology boundaries to really put something to the test. If you're familiar with Ruby on Rails, I created this very simplistic (rails 3) client class using the HTTparty gem (don't forget to add it to your gems file) to be able to consume the project search service.
require 'rubygems'
require 'httparty'

class ProjectResource
  include HTTParty
  base_uri 'http://localhost:8080/jaxrstest/REST/project'
  format :plain
  headers 'Accept' => 'application/json'

  
  def self.search_projects(name)
    
    #this performs a GET request to the project search, returning the JSON data as plain text. That way we can easily use ActiveSupport to deserialize the data into model class objects.
    data = get("/search/#{name}")
    projects = ActiveSupport::JSON.decode(data)
    return projects
  end  
end
Where 'projects' is then an array of Project objects, which are simple Rails model classes that match the Java ones above. The fact that they're ActiveRecord model classes is what makes it possible for ActiveSupport to magically translate the JSON data into the appropriate model types without needing any hint at all. As an example, here is a stripped version of the Project model class:
class Project < ActiveRecord::Base
  
  attr_accessible :id, :name, :company, :company_id, :active 
  belongs_to  :company

  ...
end
How to actually setup a Rails application and use the above code is way beyond this article, but I'd just want to give an idea of how such a thing translates to another language/platform. Of course Rails being Rails there are likely dozens of alternatives to be able to consume a web service; I picked this route because it is basic and lightweight.

You don't have to write any client to test this out, you could also install the excellent HttpRequester Firefox plugin, which allows you to fire very specific HTTP requests to an url. That way you can fire a GET request with the accept header set to application/json. If you're not a Firefox user I'm sure there is a similar plugin for your browser of choice, or you could create a simple java command line thing using the Apache HttpClient API.

Producing multiple content-types

So far we've locked our service down to JSON, but what if some clients must remain in the stone age and demand XML? Easy enough, if you look in the Jersey reference manual you'll quickly spot that the content-type for XML is "application/xml". But what if we wanted to support both json AND xml output? It would really stink if we had to duplicate each service call. Luckily you don't have to, we can augment our service call as follows:
  @GET @Path("/search/{name}")
  @Produces({"application/json", "application/xml"})  
  public Project[] search(@PathParam("name") String name) {

This one method can now service both JSON and XML data. But which one is going to be the default? Jersey at least, it seems, favors XML. If you navigate to the service call in a browser you'll now get XML output as a result. Our Ruby client as defined above however, provides an accept HTTP header of 'application/json'. That accept header makes it so a call to the same URL will in fact produce JSON data as a result!

Reversing the flow: pushing data

Until now the article has been dealing with one way to provide data to a client. Now lets examine the other way around: how does a client push data to our service? Since this is fully HTTP based, the answer lies in the request method. Until now we've been dealing with GET requests, but there is also HTTP POST, PUT and DELETE. Delete won't need any explanation, but POST and PUT are subject to a certain convention you'll find common in RESTful services - POST is used to add something, PUT is used to update something.

If you've ever built any kind of web application you've likely built a web form which is submitted to the server through a POST request; that same web form can likely be used to either add or update a record. Now there is nothing stopping you from creating a RESTful service which reacts to a POST request and both inserts and updates data; there is nothing stopping you from using PUT in stead. But when it comes to providing a service, you want to lay down a very precise and clearly defined interface or contract - it is not a bad idea at all to actually separate insertion and modification into two separate methods and following the convention for one very simple reason: the ability to make safe assumptions which is the biggest step towards self-documenting code.

Okay, lets create a method to add a company to our system. Since I've been pushing JSON until now, let's in fact make our client offer the data in JSON format.
  @POST @Path("/company/add")
  public void createCompany(Company comp) {
    // normally you'd do something like use JPA to persist the company to the database.
    System.out.println("CREATING COMPANY WITH NAME: " + comp.getName());
  }

This method is actually more flexible than you'd think as the client can transmit the data both in JSON and XML format and both will work; if you'd want to lock the service down to a specific type then you could add a @Consumes annotation to the code.

Now this will certainly work, but for our client it lacks critical feedback. The company is now stored, but how does one uniquely identify this company to for example be able to fetch it later or bind an employee to it? Much like a GET method, we can return something from our POST method. You could echo back the entire Company object but for example with an ID property added, but a more clever solution is to return some sort of result object which can also communicate success or failure.
@XmlRootElement
public class IdResult {

  private long id;
  private boolean ok;
  private String msg;
  
  public IdResult(){
  }
  
  public IdResult(long id){
    this.id = id;
    this.ok = true;
  }
  
  public IdResult(String msg){
    this.ok = false;
    this.id = -1;
    this.msg = msg;
  }

  // getters and setters here
}

With that simple object we can now give proper feedback to the client:
  @POST @Path("/company/add")
  @Produces("application/json")
  public IdResult createCompany(Company comp) {
    // normally you'd do something like use JPA to persist the company to the database. We need an ID to return, so just generate something.
    System.out.println("CREATING COMPANY WITH NAME: " + comp.getName());
    return new IdResult(System.currentTimeMillis());
  }

Time to try that out. With the path set the full url would be http://localhost:8080/jaxrstest/REST/project/company/add; if you put that in a browser you should get a "method not allowed" error message (since we're sending a GET request to something which expects a POST request). To do a proper test you could create a simple client method or by using that HttpRequester firefox plugin I mentioned earlier. Use the following POST request body, which is our company in JSON format sans employees:
{"id":0,"name":"My super duper company"}

To be able to actually send this POST request you have to make sure that you are setting the content-type of the request to application/json! As an accept header you have to set application/json as well since that is what the service call @Produces. the output of such a request would now be like this:
{"id":"1341924925250","ok":"true"}

Lets add another Ruby on Rails example here just to keep the article consequent. Since we have to set an explicit content-type, its best to put update code in a separate resource class specifically aimed at doing update actions. To get the result you could define another IdResult model class and deserialize the JSON response like in the previous client example, but to be original lets skip that and simply let HTTparty do some work for us. It can actually deal with JSON response data and return the individual fields as a hash.
class ProjectUpdateResource
  include HTTParty
  base_uri 'http://localhost:8080/jaxrstest/REST/project'
  format :json
  headers 'Accept' => 'application/json'
  headers 'Content-Type' => 'application/json'

  def self.add_company(name)
    
    #construct a company model object and serialize that to JSON (limiting to specific fields)
    company = Company.new(:id => 0, :name => name)
    data = company.to_json(:only => [:id, :name])

    #post the JSON data as the body of the request and get the response
    res = post("/company/add", :body => data)
    
    #get the individual fields from the response
    ok = res['ok'] == 'true'
    id = res['id']
    msg = res['msg']
    
    #should do some error checking here and throw an exception where necessary.
    puts "----------------------------- OK: #{ok}, ID: #{id}, MSG: #{msg}"
    return id
  end
end
Again a very basic and lightweight way to do it and there are probably many alternatives to do it in a more RESTful way. It doesn't matter, what matters is that we get the data to our service in a way that is expected and we can read out the response in an easy way. At this point you may run into many HTTP errors, all of them caused by the content-type and the accept headers of the client not matching with what the services produces and consumes. Its a question of fiddling until something works.

Working with dates in your models

Dates are a difficult thing, because they come in so many shapes, sizes and orders. Therefore marshalling/unmarshalling a Date is not something that can just happen; likely you will want to impose a little control over this. The easiest way of all is to just use dates in textual format, but if someone sends you an invalid date you can't let the API deal with that, you have to parse and validate yourself. My choice to attack this problem is to use a JAXB adapter.

Watch out though!: older versions of Jackson (the JSON mapping library often used in JAX-RS implementations) did not handle the adapters at all. Make sure you are using an up to date version of Jackson / your JAX-RS implementation. If you're a JBoss 7.1 user, check the start of this article for upgrade steps.

First of all we need our adapter, which is as simple as this:
public class DateAdapter extends XmlAdapter<String, Date> {

    @Override
    public Date unmarshal(String v) throws Exception {
        SimpleDateFormat sdf = new SimpleDateFormat("dd-MM-yyyy");
        return sdf.parse(v);
    }

    @Override
    public String marshal(Date v) throws Exception {
        SimpleDateFormat sdf = new SimpleDateFormat("dd-MM-yyyy");
        return sdf.format(v);
    }
}

This class will work with dates in Dutch format, adapt the pattern to what you need. Now we need to tell our JAX-RS implementation that we want to use this adapter to marshall/unmarshall dates. Lets take a more extreme example and you are using JPA entities as your model classes (just to show that you can), it could turn into something like this:
@Entity // JPA
@Table(name="person") // JPA
@XmlRootElement // JAX-RS
public class Person {

  @Id // JPA
  @GeneratedValue // JPA
  private int id;

  private String name;

  @Temporal(TemporalType.DATE) // JPA
  @XmlJavaTypeAdapter(DateAdapter.class) // JAX-RS
  private Date birthdate;

  // getters and setters
}
Its a bit annoying; I have not found a way to set an adapter globally yet but I'm sure there is a way to achieve this result in some way...

Dealing with CORS

CORS, or "Cross Origin Resource Sharing" is a protection mechanism built into modern browsers that basically prohibits websites from making ajax requests to just any other server; by default javascript is only allowed to make requests to the host (and port) from which the resource was serviced. That's a bit limited especially when you want to make ajax requests to a RESTful service which will almost never be the host which services the web application.

To be able to get around that you need to setup CORS on your server and it is surprisingly simple on Tomcat. Simply make sure that your Tomcat version is up to date (IE. if you run Tomcat 7, make sure you are running the latest one) and then simply configure this filter in your web.xml:
  
    CorsFilter
    org.apache.catalina.filters.CorsFilter
  
  
    CorsFilter
    /REST/*
  
Be sure to match the URL pattern to whatever you URL pattern you have mapped your RESTful service too. If you are not using Tomcat, you'll have to dig into your server's manual to figure out how to properly setup CORS, or possibly manually add the headers to the response with your own filter. For further reading when using Tomcat: the documentation page.

Is it really a good idea to mix JPA and JAX-RS?

Int this article I've shown a few examples where JPA entities are exposed in the webservice calls. That is certainly very useful as it prevents duplicate code, but is it really a good idea?

The answer is unfortunately: absolutely no. There are a myriad of problems attached to exposing the database layer directly in the webservice interface, as that is what you're doing. Lets list a few:
  • Any field you add, change or remove in an entity directly affects and changes the webservice contract!
  • You need to actively exclude properties from being serialized; all are exposed to the outside world by default.
  • JAX-RS serializes entities, so all their lazy properties will be hit

What it boils down to is that you have absolutely no freedom and you severely lock down your capabilities to make any changes to the data model of the application. Basically if an entity is exposed to the outside world, it better stay exactly the way it is. If you change it, any client depending on its old state will likely break unless it is modified to match the new serialized state.

You do not want that head ache. You should use POJO value objects which expose the minimum amount of information. If you then change an entity you don't necessarily have to change the VOs that match it - you only change how the data in the VOs is fetched from the entity. Separation of concerns - it won't lead to less code, but it will prevent burn out.

Securing our web service

One day I'll provide exact steps on how to configure Tomcat 7 to do such things as utilizing HTTPS and how to setup HTTP digest authentication. But that will require further experimentation on my part as well.

Time to call it a day

Not an entire manual on setting up JAX-RS services but not hello world either. I hope with this article I've given you the leg up that makes you productive with JAX-RS as quickly as possible. For further reading I suggest you get a good book on the API, but you may also want to check out the JEE 6 tutorial chapter on JAX-RS. For Jersey specific documentation there is the Jersey reference manual. And I'd like to repeat once more that if you were looking for documentation on creating a client using JAX-RS, I have a whole article dedicated to that topic.

Good luck with your project!

Monday, July 9, 2012

JAX-RS client: beyond hello world

So, RESTful webservices. You came here, so you must be interested in invoking a RESTful webservice from a Java client. At this point you must have also figured out that JAX-RS is in fact the way to go when it comes to invoking RESTful webservices, even though technically you could do it with a simple URLConnection or a little more robust: the Apache HttpClient API. Of course JAX-RS offers some benefits, the biggest one being that it takes a big pile of work away from you.

Before you can use JAX-RS, you must first understand JAX-RS. The primary thing to understand is that it is only a specification: it doesn't actually do anything. You can't download JAX-RS and expect to be making connections to servers. To actually be able to do anything you need something that implements the JAX-RS specification and in true Java fashion, you have a plethora of choices. Third party offerings include CXF, Restlet and RESTEasy - Oracle delivers a reference implementation called Jersey; that's what I'll be using for the purpose of this article but since its a specification, it doesn't really matter which implementation you're using; the difference will mostly be in the setup part. If you deploy the client as part of a JEE 6+ application, a JAX-RS implementation is already provided by the container.

Also be aware that Jersey is already an API with a development history; it has grown up a lot since its 1.0 days with plenty of added features and improvements in the way things are done; some of which are not exactly documented clearly yet. Plenty of articles and blogs on the net will refer to how things were done in the earlier days; do keep an eye out for the date of a posted article to know how accurate the information in it is going to be. For this article I made an effort to figure out what is the more up to date way to use the API (at the time of writing of course), but perhaps I missed something.

The Jersey website actually hosts a somewhat useful reference manual; I say somewhat because like most reference manuals its a document without focus and therefore hard to follow in that "I'm new, how do I begin" state that we all begin in. So it didn't stop me from having to hunt through many a forum to get to the point where I could actually effectively put JAX-RS to work for me. Most articles I found on the net of course went no further than the bare basics; how do you setup a hello world service, how do you create a hello world client. Well not me, I'm diving a bit deeper here. To not make the article too complex I'm only focusing on creating a RESTful client here. The server/service part of it I have put in a dedicated article (because again: had to hunt through forums, make guesses and do plenty of experiments to get it to work the way I wanted).

What this article will cover is the following:
  • Maven setup / dependencies
  • how to connect to a server
  • how to invoke a service and get the results - with nested objects
  • how to deal with collections and arrays
  • how to post data to a service
  • performance enhancement
  • dealing with authentication

Then there is the question of protocol: what data format to use? The most common ones are XML, YAML and JSON - I'll be going for the one that makes the most sense to me, which is JSON.

Before we begin I'd like to stress one thing: I'm not a JAX-RS expert. This article is in no way in its final form and will continue to grow much like most of my articles expand and improve over time as I continue to learn from my own mistakes. Feel free to point out glaring mistakes or omissions in the comments; I actively maintain my articles so you can be quite sure that mistakes will be fixed.

Setting up Maven

To get going with Jersey you need only a few dependencies which are thankfully in Maven central. To keep it basic lets assume you want to deploy to Tomcat and not a full JEE application server. You would declare your dependencies as follows:

    <dependency>
      <groupId>com.sun.jersey</groupId>
      <artifactId>jersey-client</artifactId>
      <version>1.18.1</version>
    </dependency>
    <dependency>
      <groupId>com.sun.jersey</groupId>
      <artifactId>jersey-json</artifactId>
      <version>1.18.1</version>
    </dependency> 

To know the latest version of Jersey, simply consult Maven central search or here: Jersey in Maven central

If you are deploying to for example Glassfish 3 or JBoss 7 you would mark these dependencies as provided, since all required dependencies are already there in the container. You will probably want to figure out which JAX-RS implementation is delivered with your application server so you can make your compile time dependencies match up.

That's it for the client (to also provide a RESTful service, you would include the jersey-server artifact). Of course Maven being Maven, this will provide you with a whole host of transitive dependencies. For the people who don't use Maven (please, come to the dark side) the following list of dependencies are actually going into the webapp with the Jersey 1.18 version:
  • jersey-core 1.18.1
  • jersey-client 1.18.1
  • jersey-json 1.18.1
  • jettison 1.1
  • activation 1.1
  • stax-api 1.0.2
  • jaxb-api 2.2.2
  • jaxb-impl 2.2.3-1
  • jackson-core-asl 1.9.2
  • jackson-jaxrs 1.9.2
  • jackson-mapper 1.9.2
  • jackson-xc 1.9.2
Note that both JAXB and Jackson are in this list. Both of these APIs are used by Jersey to do mapping of data to objects; JAXB for XML and Jackson for JSON. For ease of deployment JAXB annotations can be used on your pojos to provide the mapping information no matter which data format is used; Jersey will still use Jackson under the hood to do the actual mapping work.

What about a server?

To be able to build a client you need a server that can be invoked, at least to test stuff out. Unfortunately that is a bit of a challenge posed to me in this here article. But in order to understand how the client works you don't really need a server; you only need what the server outputs which in our case is a blob of JSON formatted data. So without further ado lets define that exact blob.

Somewhere in the world we have a service that allows us to search for projects. A project has basic properties such as a name and a deadline, but it is also tied to a company. A company on its own has a name and an address and a list of employees. Sounds pretty basic, but in that one example we cover nested objects and a collection.

In JSON terms, that would translate to the following which I'll pretty print for the purpose of making it possible to see what's going on (in a normal circumstance you don't want unnecessary whitespace in your datastream):

[{
  "active":true,
  "id":1,
  "name":"Razorblade supershave mark 2",
  "company":{
             "id":1,
             "name":"Razorblade Ent.",
             "employees":[{"id":1,"name":"harry"},{"id":2,"name":"sally"}]
            }
 },
 {
   "active":true,
   "id":9,
   "name":"fortune cookie dispenser",
   "company":{
              "id":2,
              "name":"Unfortunate Inc.",
              "employees":[{"id":5,"name":"gordon f."},{"id":7,"name":"sam stone"}]
             }
 },
 { "active":true,
   "id":15,
   "name":"fortune cookie baking device",
   "company":{
              "id":2,
              "name":"Unfortunate Inc.",
              "employees":[{"id":5,"name":"gordon f."},{"id":7,"name":"sam stone"}]
             }
 }]

Consider this a full dump of the database; there are only three projects in here. This is not typed by hand by the way - this is the output of a Ruby On Rails service that I created for testing purposes. The fact that I could whip up my Java client and it "just worked" is what really made me go "WOW!", that hasn't happened a very long time. In my JAX-RS server article I'll be basically creating a simplified Java version of the service that produces the exact same output.

If you're not familiar with JSON I urge you to go read up on it a bit, but the above should still be human readable for a programmer. [] represents an array of elements, {} represents the start of a complex type which in Java terms will translate into a POJO bean. Quite easy to read, not as easy to type.

Defining the model

I mentioned POJO beans. To be able to read out the above JSON data we'll need a model to be able to store the information. Three complex types makes three Java POJO classes. I'll leave out the getters and setters to save some space.

@XmlRootElement
public class Project {

  private boolean active;
  private long id;
  private String name;

  private Company company;

  // getters and setters here
}


@XmlRootElement
public class Company {
  private long id;
  private String name;
  private Employee[] employees;

  // getters and setters
}

@XmlRootElement
public class Employee {
  private long id;
  private String name;

  // getters and setters
}

Only a few things to note:
  • the XmlRootElement annotation; that is a JAXB annotation that basically marks the pojo class as a complex type to be used in mapping the data; either JSON or XML.
  • the properties of the beans match the labels as you'll find them in the JSON data
  • array of employees in the JSON data, array of employees in the Java class. You can however replace the array type with a List<Employee> type if you so please, it magically works.
Convention over configuration: you're best off just using the exact same property field names as you find them in the JSON data, or else you'll have no choice but to add extra annotations to provide mapping names. With our model in place we can now go ahead and write our surprisingly simple client call.

Invoking the call

Now we get to the interesting part. The webservice we're talking with provides a service which allows us to search for projects by a (partial) name. The imaginary RESTful url to invoke is:

http://www.superpeople.com/REST/

Under that URL are all the different services, of which our project search which can be invoked as follows through a GET request:

http://www.superpeople.com/REST/project/search/:NAME

In that url :NAME is to be replaced with our partial project name. The services is structured such that all project related functions are under the /REST/project/ url. Lets define the simplest way to invoke this service and get the results without worrying too much about performance just yet. This is done in only three steps:

1) obtain a JAX-RS Client instance
2) create a WebResource using this Client to the URL you want to invoke
3) actually invoke it, optionally setting some parameters

Creating a Client is very little work and is pretty much boilerplate code. There are multiple ways to actually create a client, I picked this as the "way I'll always do it":

  public Client createClient(){
    
    ClientConfig clientConfig = new DefaultClientConfig();
    clientConfig.getFeatures().put(JSONConfiguration.FEATURE_POJO_MAPPING, Boolean.TRUE);
    clientConfig.getProperties().put(ClientConfig.PROPERTY_FOLLOW_REDIRECTS, true);    
          
    Client client = Client.create(clientConfig);
    return client;
  }

The ClientConfig class allows for some basic properties to be set; most of which you will likely not ever have to deal with. The important one here is the FEATURE_POJO_MAPPING setting: this tells the JAX-RS client that we'll be dealing with JSON data.

Okay on to the actual web invocation.

 
public static final String HOST = "http://www.superpeople.com/REST/"; 
public static final String PROJECT_URL = HOST + "project/";
public static final String PROJECT_SEARCH = "search";

public Project[] searchProjects(String name){
  
  Client client = createClient();
  WebResource res = client.resource(PROJECT_URL);

  return res.path(PROJECT_SEARCH + "/" + name)
            .accept(MediaType.APPLICATION_JSON_TYPE)
            .get(Project[].class);
}

The first thing that may seem off to you are those constants which are hardcoded; later on we'll make that a little more robust and a little less hardcoded but for now lets keep it simple. Basically we split the resource up into two parts: the 'project' resource and the 'search' function. As you can see a WebResource is constructed for the project resource. Later on we'll see why this is good design, but for now take good notice of that path() method being called; there we pass in the actual function that we want to invoke plus any variable parameters that the service call may provide. In our example the search demands a partial project name to be passed, so that's what we dump into the path.

Let's say that we search for 'fortune'; the full URL invoked will then become:

http://www.superpeople.com/REST/project/search/fortune

The imaginary service will then do a search for projects with a name starting with the word 'fortune';which in the case of our massive three project database should return two results.

Moving along to the accept method, that shouldn't need much imaginative power. Here we declare that the service should expect JSON data.

Finally the method that does all the work: get. Upon this call the restful service is actually invoked and the results will be translated into that which we pass into get() as a parameter; in this case an array of Project objects. And that's where all you have seen so far comes together:

- our JSON data structure as delivered by the phantom web service
- the pojo classes with the minimal annotations that matches said JSON data structure
- the JSON support we activated through the ClientConfig settings

All that equals magic happening during the get() invocation (because this is a GET request; if it would be a POST request you would use the post() method). Jersey will at this point read the JSON data and under water use the Jackson API to translate that data into the object hierarchy we've built, including mapping to child objects and even our employee array in the Company object; it just works. And if it doesn't work then you'll get a very clear error stating where the typo is.

Of course even though magic is being performed, its still magic under your control. YOU have to define the proper pojo object structure with proper naming. YOU have to know the exact URL layout of the service to invoke. YOU have to know in what way data will be returned (single object, array of primitives, array of objects). The tool is not doing the work for you - it is making your work easier. Exactly how it should be.

Now we've built our client call to return an array of projects. But what if you wanted a list of projects in stead? Simply swapping out the array type with a List type doesn't cut it in this case, this is how you do that:

  return res.path(PROJECT_SEARCH + "/" + name)
            .accept(MediaType.APPLICATION_JSON_TYPE)
            .get(new GenericType<List<Project>>() {});


The GenericType object is there to help the translation to the proper generic List type in such a way that the compiler likes it. From now on, it's copy/paste logic.

URL parameters

Let's say the service definition is actually that you invoke the URL like this:

http://www.superpeople.com/REST/project/search?name=fortune

The only thing that changes is the way we do the invocation.

  return res.path(PROJECT_SEARCH)
            .accept(MediaType.APPLICATION_JSON_TYPE)
            .queryParam("name", "value")
            .get(Project[].class);

If you have multiple query parameters then simply call queryParam() multiple times. Alternatively you could use this too:

  MultivaluedMap<String, String> params = MultivaluedMapImpl();
  params.add("name", name);

  return res.path(PROJECT_SEARCH)
            .accept(MediaType.APPLICATION_JSON_TYPE)
            .queryParams(params)
            .get(Project[].class);

Which would allow you to set multiple parameters in one go, at the expense of more boring boilerplate code.

Time to put that to the test

You're now already at the point where you can try it out. Since you are interested in creating a JAX-RS client, you must have some sort of RESTful service that you want to be able to invoke. If you do not - well you can always try to create one for example by following my server article.

But of course where there is a will to be lazy, there is a way to be lazy. You don't need any service; the only thing you need is a way to do a GET request to some text file which will literally return the JSON data that I provided earlier. Paste that in a file test.json and make sure you get get to it through a web url even if it is on the localhost. Then use the following bit of code to use it:

  WebResource res = client.resource("http://localhost:8080/test.json");

  return res.accept(MediaType.APPLICATION_JSON_TYPE)
            .get(Project[].class);

It should work just fine even in the pretty printed format.

The reverse situation: sending data

Until now we've been leeching information from our phantom service. What if we'd actually want to return something? Like create a new company to which projects and employees can be bound?

That is in fact quite simple but it requires a little theory. Until now we've been dealing only with GET requests; in other words requests that fetch something. To do modifications we need to use the other HTTP request methods: POST, PUT and DELETE. RESTful webservices tend to define a convention: POST is used to add information, PUT is used to update information. I hope I don't have to explain what delete will be used for :)

So in order for our client to be able to send something we first need the contract; our phantom service requires us to POST JSON data to the service url /project/company/add and will in return give a JSON complex type as an answer in the form of this IdResult object:
@XmlRootElement
public class IdResult {

  private long id;
  private boolean ok;
  private String msg;
  
  public IdResult(){
  }
  
  // getters and setters here
}

Which in JSON terms may look something like this for a success result:
{"id":"1341924925250","ok":"true"}

With that knowledge and the IdResult object at hand, we can construct our client call.
public static final String COMPANY_ADD = "company/add";

public IdResult addCompany(String name){
  
  Client client = createClient();
  WebResource res = client.resource(PROJECT_URL);

  Company company = new Company();
  company.setName(name);

  return res.path(COMPANY_ADD)
            .accept(MediaType.APPLICATION_JSON_TYPE)
            .post(IdResult.class, company);
}

And really, that's all there is to it. JAX-RS takes care of translating our Company object to the JSON body of the POST request and will also translate the JSON response sent back by the service into the IdResult object through that one post() invocation. Powerful stuff huh?

Making it better: stop hardcoding stuff

In the code I gave you until now the actual host of the service was embedded in the code. That may work for development and testing purposes, but if you put a REST client in production you will want to be able to manage the actual host URL outside of the code; for example you may have a completely different host URL for test/acceptance and production environments, in which case it depends on where the application is deployed which URL is to be used.

How to actually configure the URL depends on what you have available. Some options:
  • Pass it as a system property to the java command (-Dresturl=...)
  • In case of JBoss: define it in the property service
  • Store it in a settings database table
There are probably more solutions to be found but I believe in the power of no more than three and lets be honest: the first impulse anyone is going to have is store it in a database. My JSF 2 and JPA 2 on Tomcat 7 article can help you to write the code to do exactly that.

Making it better: re-use objects

The code as it is now has you creating a Client and a WebResource each time you do a call. These are quite expensive operations and as it turns out: you need to create them only once. We're talking about a stateless design here, you can safely re-use both the Client and the WebResource class in multiple threads without risk of collision. So lets do exactly that. First of all, lets isolate the actual initialization. How to design the code is pretty much a personal thing. I like to have setup in one place, so I tend to create an abstract base class in which I deal with it. Something like this:
public abstract class RestBase {

  public static final String PROJECT_URL = "/project/";
  
  protected static Client client = null;
  protected static WebResource PROJECT_RESOURCE = null;
  
  public static void init() {
    
     ClientConfig clientConfig = new DefaultClientConfig();
     clientConfig.getFeatures().put(JSONConfiguration.FEATURE_POJO_MAPPING, Boolean.TRUE);
     clientConfig.getProperties().put(ClientConfig.PROPERTY_FOLLOW_REDIRECTS, true);    
          
     client = Client.create(clientConfig);
     
     // whatever database logic you have to get the host name from the database
     String hostname = SettingsDao.getSetting("REST_host");

     PROJECT_RESOURCE = c.resource(hostname + PROJECT_URL);
     
     // initialize other resources here
  }
}

This is the base class for all RESTful client services I'll be creating (which just happens to copy/paste very nicely). It has a method to initialize the one Client object we'll be exposing On top of that it initializes all the RESTful resources in one go and exposes them to subclasses. What's with all the statics I hear you wonder!? I know, it's not object oriented design. But so what? This code at least is easy to use, as we'll see in a moment. Next up is the project client service:
public class RestProject extends RestBase {

  public static final String PROJECT_SEARCH   = "search";
  public static final String PROJECT_GET      = "get";

  public static List getProjects(String name) {

    return PROJECT_RESOURCE
            .path(PROJECT_SEARCH + "/" + name)
            .accept(MediaType.APPLICATION_JSON_TYPE)
            .get(new GenericType<List<Project>>() {});
  }

  public static Project getProject(long pid) {

    return PROJECT_RESOURCE
            .path(PROJECT_GET + "/" + pid)
            .accept(MediaType.APPLICATION_JSON_TYPE)
            .get(Project.class);
  }

Huuuuu, more statics. You may not like that, in which case I challenge you to change it to your liking. Me? I think 100% stateless design is only hindered by the need for object creation before you can actually invoke something.

Note how I added a second phantom call just to make a little more exciting. The JSON data that this call may return to match our client code will look like this:

{
  "active":true,
  "id":1,
  "name":"Razorblade supershave mark 2",
  "company":{
             "id":1,
             "name":"Razorblade Ent.",
             "employees":[{"id":1,"name":"harry"},{"id":2,"name":"sally"}]
            }
}

So only one "complex type" which maps to our Project pojo class. As you can see the code manages that by simply passing in Project.class to the get() method. You know, for once this is an API that is really cleverly designed.

Now you also see the benefit of splitting the project resource from the actual functions that are exposed through the project URL; the same 'project' web resource can then be re-used to call the different functions ('search' and 'get' in this case). That minimizes the amount of bootstrapping that needs to be done.

Speaking of bootstrapping: that init method still needs invoking. A context listener can help us do that. The listener will be invoked upon deployment of our web application.

@WebListener  
public class InitContextListener implements ServletContextListener {
  
  private static volatile boolean initialized = false;
  
  public void contextDestroyed(ServletContextEvent ev) {
    
  }

  public void contextInitialized(ServletContextEvent ev) {

    if(initialized == false){

      RestBase.init();
      initialized = true;
    }
  }
}

And there you have it. Now as I said most of this is subject to personal taste. You may in fact skip the RestBase class altogether and simply initialize a Client instance per client service class you create. You may move all the setup logic in RestBase directly into the InitContextListener. If performance is not much of an issue because you do only a handful of calls sporadically, you may just want to use the "simple" way of doing things that the article began with. Don't re-use anything, just initialize on the spot, use it and discard it.

Whatever you feel comfortable with, and I do urge you to think about it and experiment a bit with it to find that which is comfortable to you (and your colleagues).

You can also simply run as a command line application at this point, assuming you have your project dependencies properly setup (goes pretty much automatic when you have a Mavenized project).

public static void main(String[] args){
  
    RestBase.init();
    
    List<Project> projects = RestProject.getProjects("fortune");
    System.out.println("Got " + projects.size() + " projects!");  
}


Plugging in Apache HttpClient

By default Jersey will simply use the JDK features to do HTTP calls, which boils down to the usage of HttpURLConnection and HttpsURLConnection. For simple services this will be sufficient although people have been known to experience sporadic failures when using these classes in combination with certain (unstable?) web servers. No matter since these classes have been basically made obsolete by the incredibly powerful, feature complete, fast and easy to use HttpClient built by our friends at Apache. So is there a way we can make Jersey use it?

Yes we can, through the following setup code. First of all an additional dependency for our Maven pom:

<dependency>
    <groupId>com.sun.jersey.contribs</groupId>
    <artifactId>jersey-apache-client</artifactId>
    <version>1.18.1</version>
</dependency>

For the non-maven users, this adds the following additional dependencies to the project:
  • jersey-apache-client 1.18.1
  • commons-httpclient 3.1
  • commons-logging 1.0.4
  • commons-codec 1.2
With that dependency in place, you can construct the client differently like this:

Client c = ApacheHttpClient.create(clientConfig);

And then you've succesfully plugged in HttpClient in place of the less reliable JDK URLConnection APIs. At this point you may also want to activate automatic cookie handling support which is one of the more useful bonuses that HttpClient gives you:
  ClientConfig clientConfig = new DefaultClientConfig();
  clientConfig.getFeatures().put(JSONConfiguration.FEATURE_POJO_MAPPING, Boolean.TRUE);
  clientConfig.getProperties().put(ClientConfig.PROPERTY_FOLLOW_REDIRECTS, true);    
  clientConfig.getProperties().put(ApacheHttpClientConfig.PROPERTY_HANDLE_COOKIES, true); 
  
  Client c = ApacheHttpClient.create(clientConfig);

And that's basically all there is to it; try running whatever test code you have created and you'll see that things still work as they did before. In my experience: a bit faster even. The downside of using HttpClient is that you are of course weighing down your application with even more dependencies.

Dealing with HTTPS

One day this article will have a step-by-step guide on how to not only write the code but also setup a test environment. Until then I defer to this most excellent stackoverflow thread which should be the kick in the right direction that you need:

http://stackoverflow.com/questions/3434309/accessing-secure-restful-web-services-using-jersey-client

Authentication

Authentication can be solved at different levels. First of all there is the API level; the service itself may have some sort of authentication scheme built in, for example requiring you to pass along a HTTP header or certain parameters that identify who is calling. Sending a HTTP header is easy, like this:

    return PROJECT_RESOURCE
            .header("NAME", "VALUE")
            .path(PROJECT_SEARCH + "/" + name)
            .accept(MediaType.APPLICATION_JSON_TYPE)
            .get(new GenericType<List<Project>>() {});

The header() method does all the work. Invoke it multiple times to add more headers.

It may be that the call is actually secured based on IP filtering. Then there is nothing to do with or in code, the firewall of the service provider will either allow or disallow calls from your network. To gain access the firewall needs to be configured appropriately; good luck dealing with tech support.

It may also be that a form of HTTP authentication is employed. Contrary to what the online documentation will try to make you believe, setting up something like basic HTTP authentication through JAX-RS is really as simple as this:

client.addFilter(new HTTPBasicAuthFilter("username", "password"));

And more secure digest authentication:
client.addFilter(new HTTPDigestAuthFilter("username", "password"));

This works just fine with either the "default" Client or the Apache HttpClient.

Finally there is OAUTH, which is an authentication solution and API that has become somewhat of a standard on the net through popularity. Unfortunately I have no experience with using the API just yet so I cannot document it, but some initial scouting on the net seems to indicate that OAUTH support for Jersey is actually available.

Time to call it a day

There you have it. Less than an entire manual on JAX-RS, more than hello world. I hope in this one article I managed to capture the more common use cases for JAX-RS on the client side of things and you are now well on your way to actually being productive. At least I was after having pieced all this stuff together.

For further reading I suggest you get a decent book on the API, but you may also want to check out the JEE 6 tutorial chapter on it, which is likely going to be a slightly less chaotic approach to the material than the Jersey reference manual is but will focus mainly on the programming part. And let me plug my JAX-RS server article one more time :)

Good luck with your project!

Tuesday, March 6, 2012

JSF 2.1 and JPA 2 on Tomcat 7

Introduction

I'm a JBoss user and so I tend to develop against and deploy on a JBoss server (JBoss 7.1 being awesome to the max).

But you know, sometimes I look at what I have and I think to myself: do I really *need* to apply all those layers of complexity just to solve a simple problem? Think about EJBs; powerful stuff, but is it really necessary to apply it when you are building a simple webapp? Even if JEE6 makes it simple by allowing you to deploy EJBs in the war, I still don't think so. Do you really need container managed stuff? Other than a datasource: not really.

In stead of plowing ahead and bolting on everything that other smart people come up with, sometimes it is just a good thing to take a step back and return to the basics. I want to create a webapp, a simple war, do some simple JPA/Hibernate 4 database stuff and create some pages using JSF 2.1 and Primefaces (or whatever component library you fancy, if any). And I want to (hot-)deploy it on the simplest and in my opinion most stable Java servlet container of all - Apache Tomcat 7. Aha but I'm still a power user. I also want to use Maven and Eclipse. The plot thickens. Lets see what we can do...

Please be aware that I'm not aiming here to teach you how to create web applications; that is a subject that filled many books. I am not even going to make a weak attempt at it either, although I do my best to fill in as many blanks as possible. I assume you already know your stuff when it comes to web development, JSF, persistence and Maven. The main focus of this article is to bring everything together into a neat, manageable package. To do that I'm splitting the article up in three sections. First of all is the setup of tools, then we are going to get JSF up and running and something hot-deploying, finally we're going to add the database layer.


Setup tools

Of course you need to download and install Tomcat 7 and you need Eclipse. For both, download the latest releases. I'm assuming you are using at least Eclipse 3.7 SR1. We are going to be controlling Tomcat from within Eclipse, so no need to install it as a Windows service (assuming you are doing this on Windows). Just get the zip, unpack it and be almost done.

To communicate with a database you need a JDBC driver; Java databases 101. It is Tomcat that will be managing the connection pool for us, so it is Tomcat that needs access to the driver. You cannot simply deploy it with your application, it needs to be on the server classpath. To do this, put the JDBC driver in the lib subdirectory of the Tomcat installation; easy and simple. In the examples I am using HSQLDB.

Since I'm a Maven dude, that is what I'll document. All you need to do is create a simple Maven project. No need for a multi-module setup, just create a single project of type 'war'. After creation there is always the dreaded dependency setup. For now lets add the bare minimum. For more help with Maven, my article on Maven might help. Also, my Article on using Maven from Eclipse.


<dependencies>
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>1.6.4</version>
  </dependency>
  <dependency>
    <groupId>log4j</groupId>
    <artifactId>log4j</artifactId>
    <version>1.2.14</version>
  </dependency>
  <dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-log4j12</artifactId>
    <version>1.6.4</version>
  </dependency>


This solves our logging needs. I use log4j which is perhaps a bit outdated; you could also simply use java.util.logging (JUL) and save yourself a dependency. I've tried to do that myself but I've had no success so far configuring that properly so I get something as simple as a dedicated log file for my own applications.


<dependency>
    <groupId>org.apache.tomcat</groupId>
    <artifactId>tomcat-servlet-api</artifactId>
    <version>7.0.54</version>
    <scope>provided</scope>
  </dependency>
</dependencies>


These dependency allows us to compile servlet 3.0 code. Note that the servlet-api dependency is provided, as Tomcat itself already provides it for us of course.

The above dependencies will likely be the same for all web projects you do. Later we are going to add a couple more to deal with JSF and JPA. For now, lets wrap up with basic plugin configuration:


<build>
    <plugins>
      <plugin>
        <artifactId>maven-compiler-plugin</artifactId>
        <inherited>true</inherited>
        <configuration>
          <source>1.7</source>
          <target>1.7</target>
          <encoding>UTF-8</encoding>
        </configuration>
      </plugin>
      <plugin>
        <artifactId>maven-resources-plugin</artifactId>
        <configuration>
          <encoding>UTF-8</encoding>
        </configuration>
      </plugin>
    </plugins>
  </build>

At this point you will want to refresh your project settings (right click project -> maven -> update project) so that the Eclipse settings match the Maven settings (Java 7 being used). You could of course also specify Java 1.8 if you're going to be running on Java 8.

RECAP: At this point you have

- installed Tomcat 7 or higher and Eclipse 3.7 or higher
- added a JDBC driver to tomcat/lib
- created an Eclipse workspace, a maven project with war packaging and added the above into the pom
- did you check the build path, especially if there are exclusions on the resources directories? There likely are - remove those exclusions or hot-deployment is not going to work right.
- during the setup stages, continuously examine the deployment assembly; this can turn into a mess when using a mavenized project for some reason


Config files

To properly setup the application based on the choices I've made we need a pile of configuration files. Lets tackle them one at a time.

webapp/WEB-INF/web.xml
Main configuration file for a java webapp. Right now its quite bare, we'll be adding stuff to it later.


<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
      version="3.0">
       
  <display-name>testapp</display-name>
  
</web-app>  


Notice the servlet spec 3.0; Tomcat 7 provides that for us. If you would be targetting Tomcat 6 for some reason, you'd need to downgrade to servlet 2.5 (and Java 6).


resources/log4j.xml
To make log4j happy, we need to define some minimal configuration.


<?xml version="1.0" encoding="UTF-8" ?>
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">

    <appender name="ASYNC" class="org.apache.log4j.AsyncAppender">
        <appender-ref ref="CONSOLE"/>
    </appender>
        
    <appender name="CONSOLE" class="org.apache.log4j.ConsoleAppender">
        <layout class="org.apache.log4j.PatternLayout">
             <param name="ConversionPattern" value="%-d: %-5p [%8c](%F:%L) %x - %m%n"/>
        </layout>
    </appender>

    <category name="org.hibernate">
        <priority value="error"/>
    </category>

    <category name="com.yourpackage">
        <priority value="debug"/>
    </category>

    <root>
        <priority value="INFO" />
        <appender-ref ref="ASYNC"/>
    </root>

</log4j:configuration>


In the above config, replace 'com.yourpackage' with the root package of your own application. With the Hibernate filter in place the noise logging should be minimal. Note that this is aimed at development; when deploying to an actual server outside of your IDE you should add file logging to it. I refer to the log4j manual for more information.


That's it for the basic setup.

TO RECAP: at this point you will have the following files in your Mavenized project.

src/main/webapp/WEB-INF/web.xml
src/main/resources/log4j.xml

For the people who don't use Maven, there will be a summary of deployment setup at the end of the article.


Setting up hot-deployment

Before we get on to adding JSF and JPA to our little test application, you might want to go through the hot-deployment setup first. To make sure it works, lets add a very basic index.html to our webapp.

<html>
  <body>
    <h1>
Hello world</h1>
Hot deployment seems to be working.
  </body>
</html>

Now it is time to take some snapshots and show how to setup Eclipse.

First of all, we will want to create a server runtime. Choose window, preferences. In there, unfold Server and select Server runtimes. This is likely a virgin workspace, so no server runtimes will be there. Click Add to add a new one.


Simply select Apache Tomcat 7 and click next. In the next dialog, select the correct directory where you installed Tomcat 7. Click Finish to create the server runtime.


Not there yet. Close the preferences and open the servers view (window, show view, other and in the dialog select server and then servers). In the servers view, right click and select new, server.


Make sure that Tomcat 7 is again selected; Eclipse will automatically setup the runtime you just created. Click next. In the resulting dialog, make sure that your webapp is in the configured list so it will be deployed to the server. Then click finish to create the server instance; you'll notice it in the servers view and you'll also notice that your application is going to be published to the server. Hurray!



Depending on the version of Java you use, it might be a good idea to give Tomcat some more heap space; you can do that by double clicking on the server instance in the server view; an editor window opens up where you can provide fine-grained configuration for the instance. One such configuration is the launch configuration, which you can change by clicking Open launch configuration.


In the launch configuration, select the arguments tab which shows all the java properties being set. We are going to add a new one, -Xmx512m.


That gives the server a lot more breathing space should it be necessary; you can also put a lower value; during development you don't want to give yourself too much breathing space. The higher the value you set, the less chance you'll have of catching heap space related problems during development. Note that in the arguments, take good note of the wtp.deploy property being set in the launch configuration.

At this point you're ready to try it! You can try publishing the application. This will make sure that the application is deployed to the server. Of course you want to check that Eclipse is doing it right, so open up the tomcat/webapps directory and... what the heck? Nothing is deployed there!

Remember that I told you to take note of that property in the launch configuration. That directory is where Eclipse deploys the application in fact. It is going to be a path like: workspace/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/wtpwebapps. Here you'll find the deployed files. Don't meddle with these yourself, whatever you want to do: do it from Eclipse or else the published state in Eclipse may go out of sync with the filesystem. Nothing that a good clean of the project cannot fix though. In case of trouble it is good to know the location of these files so you can check just what the heck Eclipse deployed and what it didn't.


Change the deployment location

You may want Eclipse to deploy the files to a somewhat less hard to find location; I certainly do. We can configure it, but before Eclipse will allow you to to that you will need to remove your project from the list of deployables under your server instance. Right click the project name and remove it. Then, publish the server. At this point when you double click the server instance, you will be allowed to make changes to the deployment location (note: when I tried this on Eclipse Luna, I did in fact not need to remove the application from the server first).


This selection will move the deployment directory into the tomcat installation directory; much more manageable. You can choose to use the webapps folder, but you may want to just keep the one that Eclipse picked for you; it makes no difference. One thing you have to be aware of: Eclipse will change the server configuration to make this work (backups are created). That is a bit unfortunate as you can't re-use the same Tomcat instance to deploy multiple applications to. I would create a unique copy of Tomcat per application you're developing concurrently.

Whatever the case, after changing the directory you will want to make your application deployable again. You do this by simply dragging the project node in your package explorer to the server instance in the servers view. The project will reappear in the list of deployables.


Fixing the deployment settings

We are going to return to the server view one more time, this time to not look at the server settings but at the settings of the application we're deploying. Double click on the server (make sure the application is going to be deployed to it) and then click on that Modules tab hidden away at the bottom.



Here we can change two things;
  • The context name; this influence the URL under which we will be able to reach our application on our development localhost environment
  • If the webapp is reloadable or not
By default the context name will be the same as the name of the project, which should be fine. But if you have a really awkward project name, you can change it here. But what is important is to change that Reloadable property to disabled. We'll learn why later on (in case you're curious: switching this off prevents webapp-reload-on-code-change misery). Click the edit button to actually be able to change the values.

Check your Eclipse project view at this point, you will notice that Eclipse has added a 'servers' project to it, it did that when you created your server instance.


As said, Eclipse takes over the Tomcat installation and these are the files it manages for your benefit. One of these files is of interest to us at this point: the server.xml file. Open it up. If you scroll all the way to the bottom you'll find an XML element called <Context> which contains exactly the information we just input in the server dialog. Now you know where to look to validate that your settings will be correct when you start the server.



I repeat: reloadable should be false at this point.


There just has to be something wrong

Getting a project to hot-deploy properly can be a bit fiddly. Recurring problems I have is that Eclipse wants to hot-deploy test resources and wants to deploy web resources to both the root and the classes folder for some reason. When things don't go right, do a few basic things.

- make sure the Eclipse project is synced with the Maven poms (m2eclipse tends to notify you when the project is not through a build error)
- clean the project and do a manual full publish
- check the build path to make sure there are no exclusions
- check the deployment assembly; make sure no test resources are deployed and that all the elements point to the proper deployment path (and exist only once!)
- perform a clean on the server instance

Note that making changes to the build path tends to also result in automatic changes being made to the deployment assembly; make sure the two are always synced.



If all else fails, check, check and triple check all configuration files, the Maven poms, the Eclipse facets, the deployment assembly and the build path settings.Whatever problem you have, it originates from the build path and the deployment assembly, but the problem might start in a resource which influences those elements (such as a pom). Whatever the problem, use the deployed application as a guideline to figure out what is wrong. Check the files that Eclipse has copied to the Tomcat web application directory (you should have configured which one that is by now) and see what files are there. Which ones are missing? Which ones are in the wrong place? Which ones should not be there? The Tomcat documentation can guide you.


Start the server

At this point, click on the debug icon to start the server. Like when using JBoss, you can easily debug and hot-swap your code when deploying to Tomcat. Tomcat should output some basic logging to the Eclipse console, which is nearly empty right now because we're not deploying much yet. At this point, you can navigate to http://localhost:8080/testapp (or whatever context name you configured earlier) and our basic index page should pop up. Now we're ready to bolt on more stuff, beginning with JSF.


Setting up JSF 2

To work with JSF 2 you need to add only one dependency to your Maven pom:

<dependency>
    <groupId>org.glassfish</groupId>
    <artifactId>javax.faces</artifactId>
    <version>2.1.29</version>
  </dependency>

Not provided, we're deploying this baby with our webapp because its Tomcat, which does not provide JSF out of the box. When you're deploying on a full JEE application server you will want to mark this dependency as provided because JSF 2.X is already going to be provided by the container itself. Plain JSF is likely not enough for you though, you will probably want to add one of the extension frameworks. In my case, I add Primefaces.

<dependency>  
    <groupId>org.primefaces</groupId>  
    <artifactId>primefaces</artifactId>  
    <version>4.0</version>  
  </dependency>

Primefaces is nice and simple yet powerful and feature rich, and since version 4 it is also available through Maven Central so you do not need any external repository anymore.


JSF 2 - web.xml changes

JSF needs some basic setup in the web.xml so that it can start servicing our Facelets XHTML pages:

<context-param>
    <param-name>javax.faces.INTERPRET_EMPTY_STRING_SUBMITTED_VALUES_AS_NULL</param-name>
    <param-value>true</param-value>
  </context-param>
  <servlet>
    <servlet-name>Faces Servlet</servlet-name>
    <servlet-class>javax.faces.webapp.FacesServlet</servlet-class>
    <load-on-startup>1</load-on-startup>
  </servlet>
  
  <servlet-mapping>
    <servlet-name>Faces Servlet</servlet-name>
    <url-pattern>*.xhtml</url-pattern>
  </servlet-mapping>

  <context-param>
    <param-name>javax.faces.PROJECT_STAGE</param-name>
    <param-value>Development</param-value>
  </context-param>

  <!-- primefaces -->
  
  <!-- If you want to use a different theme, configure it like this
  <context-param>
   <param-name>primefaces.THEME</param-name>
   <param-value>aristo</param-value>
  </context-param>
   -->

This will setup the facelets servlet to process .xhtml files only. Note the VALUES_AS_NULL property. This setting will force that submitted values that are empty do not become empty strings but in stead are null. Lo and behold when you try that on Tomcat, it seems like the setting isn't doing anything! That's because Tomcat has a special EL implementation that also needs to be configured. We need to add another startup parameter to the server launch configuration (remember? Where you set the maximum heap space):

-Dorg.apache.el.parser.COERCE_TO_ZERO=false


The combination of those two setup steps is what makes it happen under Tomcat 7.


JSF 2 - webapp/WEB-INF/faces-config.xml file

This file has become mostly obsolete with JSF 2 and up, but it is still best to provide it even if it is empty. Some servers keep an eye on it to know when something needs to be redeployed - Tomcat is one of them. And you still need it if you need to setup very specific navigation rules.


<?xml version="1.0" encoding="UTF-8"?>
<faces-config
    xmlns="http://java.sun.com/xml/ns/javaee"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-facesconfig_2_1.xsd"
    version="2.1">

</faces-config>




JSF 2 - getting something to deploy

That's the setup you need to get to work with JSF. To prove that it works we need some basic code. A very basic facelets template (put it in the webapps folder of your project):


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
  xmlns:h="http://java.sun.com/jsf/html"
  xmlns:f="http://java.sun.com/jsf/core"
  xmlns:ui="http://java.sun.com/jsf/facelets"
  xmlns:p="http://primefaces.org/ui">
<h:head>
<title>Testapp</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
</h:head>
<h:body>
  <h:outputStylesheet target="head" library="css" name="main.css"  />
 <div id="content">
  <div id="maincontent">
   <ui:insert name="content">Page body here</ui:insert>
  </div>
 </div>
</h:body>
</html>


One thing you may notice (if you have been using JSF for longer than today) is that the outputStylesheet is in the body tag, not the head tag. This is because I added an extension framework (Primefaces) into the mix, which adds its own stylesheets. By putting the outputStylesheet in the body tag, this enforces that your own stylesheets are always added last to the page, allowing you to override styles of the framework should you be so inclined.

This uses JSF 2 resource management; save the main.css file as webapp/resources/css/main.css so JSF can find it.

Finally, lets define an index.xhtml file:


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<ui:composition xmlns="http://www.w3.org/1999/xhtml"
  xmlns:h="http://java.sun.com/jsf/html"
  xmlns:f="http://java.sun.com/jsf/core"
  xmlns:p="http://primefaces.org/ui"
  xmlns:ui="http://java.sun.com/jsf/facelets"
  template="template.xhtml">

    <ui:define name="content">
      #{index.hello}
    </ui:define>
</ui:composition>


And we need a backing bean to go with that, just to be sure that it all works:


@ManagedBean
public class Index {

  public Index(){
  }

  public String getHello(){
    return "hello";
  }
}

At this point you should remove the index.html file that you created earlier, our new index.xhtml replaces it. Check that all the new files are deployed (and the index.html you removed is no longer deployed) and then (re)start the server. At this point the startup logging should provide a hint that JSF is in fact present:

Initializing Mojarra 2.1.29 (SNAPSHOT 20120206) for context '/testapp'

Good, JSF is initializing things. This line tells you under what context name Eclipse is deploying the application which should be the context name you configured earlier. At this point navigate to http://localhost:8080/testapp and if everything is correct, you'll be greeted by the hello message from the backing bean.

TO RECAP: at this point you have the following in your project:

src/main/webapp/WEB-INF/web.xml (modified)
src/main/webapp/WEB-INF/faces-config.xml
src/main/webapp/index.xhtml
src/main/webapp/template.xhtml
src/main/webapp/resources/css/main.css
src/main/resources/log4j.xml


JPA 2 - setup

To be able to use JPA in our webapp we'll need to add a persistence provider to our maven pom. We'll be using Hibernate.

<dependency>
    <groupId>org.hibernate</groupId>
    <artifactId>hibernate-core</artifactId>
    <version>4.3.1.Final</version>
  </dependency>
  <dependency>
    <groupId>org.hibernate</groupId>
    <artifactId>hibernate-entitymanager</artifactId>
    <version>4.3.1.Final</version>
  </dependency>

This adds not only Hibernate 4.3 (check what the latest version is though) to the dependencies, but also the JPA API. Oracle does not provide an API dependency for JPA 2 (at the time of writing at least), but luckily most persistence providers do provide their own. Hibernate is not in Maven Central, you'll need to add the JBoss Nexus to your Maven configuration. My Maven article has more details on that.


JPA 2 - datasource setup

Before JPA can do anything, we need to provide a datasource. For testing purposes you don't need to install a real database, you can setup an in-memory one like H2 or HSQL. Because it is my old friend I'll be using HSQL here, but you can easily substitute it for H2 if you so please. First of all we need to add a datasource to our development setup. To do that we are going to modify another file that Eclipse is managing for us: the context.xml file (remember: its in the Servers project that Eclipse created when we setup Tomcat):

<Context>
  <Resource name="jdbc/TestappDS" auth="Container" type="javax.sql.DataSource"
            driverClassName="org.hsqldb.jdbcDriver" url="jdbc:hsqldb:mem:testapp"
            username="sa" password="" maxActive="20" maxIdle="10" maxWait="-1" />
  ...
</Context>  


Here we define a datasource/connection pool "jdbc/TestappDS", which is a simple HSQLDB in memory database. Tomcat is going to manage this datasource for us, so put the hsqldb driver jar in the tomcat/lib folder; I'll make that 100% specific: so it is NOT enough to add the JDBC driver jar to your web application, Tomcat itself needs access to it hence it goes in the Tomcat library folder. The above configuration is still not enough, we also need to hook into into our web.xml file to actually make the datasource available to our application:

<!-- tomcat -->
  <resource-ref>
      <description>DB Connection</description>
      <res-ref-name>jdbc/TestappDS</res-ref-name>
      <res-type>javax.sql.DataSource</res-type>
      <res-auth>Container</res-auth>
  </resource-ref>  




JPA 2 - resources/META-INF/persistence.xml file

Now that we have our datasource we can setup JPA and Hibernate. To make it nice and confusing, this file does not go into the META-INF folder of your webapp - it goes into a META-INF folder that is on the classpath. In maven terms that means that you create the file in src/main/resources/META-INF/persistence.xml so it ends up in webapp/classes/META-INF/persistence.xml.


<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
    http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd" version="2.0">

  <persistence-unit name="testapp" transaction-type="RESOURCE_LOCAL">
    <provider>org.hibernate.ejb.HibernatePersistence</provider>
    <non-jta-data-source>java:/comp/env/jdbc/TestappDS</non-jta-data-source>
    <properties>
      <property name="hibernate.cache.provider_class" value="org.hibernate.cache.HashtableCacheProvider" />
      <property name="hibernate.format_sql" value="true" />
      <property name="hibernate.show_sql" value="false" />
      <property name="hibernate.jdbc.batch_size" value="20" />
      <property name="hibernate.cache.use_query_cache" value="false" />
      <property name="hibernate.hbm2ddl.auto" value="update" />
      <property name="hibernate.cache.use_second_level_cache" value="false"/>
    </properties>
  </persistence-unit>

</persistence>


Tomcat is not a JEE container, so we are not using JTA or container managed transactions. In stead it is a simple RESOURCE_LOCAL setup. Fortunately, we don't need to declare any entities here; they are still discovered during the setup of the EntityManagerFactory. Hibernate is setup here to auto-generate the database from the entities, which is the most convenient way to work with an in-memory database.

Also note the JNDI name of the datasource. That is as it is documented in the Tomcat datasource HOW-TO.


TO RECAP: right now you'll have the following in your project:

Servers/context.xml (modified)
src/main/webapp/WEB-INF/web.xml (modified)
src/main/webapp/WEB-INF/faces-config.xml
src/main/webapp/index.xhtml
src/main/webapp/template.xhtml
src/main/webapp/resources/css/main.css
src/main/resources/META-INF/persistence.xml
src/main/resources/log4j.xml

And you put a JDBC driver jar of whatever database you are going to use in the tomcat/lib folder.

It might be interesting for people who do not use Maven to know where all the configuration files need to go in the war:

WhatWhere
web.xmlWEB-INF
faces-config.xmlWEB-INF
persistence.xmlWEB-INF/classes/META-INF
log4j.xmlWEB-INF/classes



JPA 2 - Managing entities and transactions

Normally the EJB container would take care of doing such things as maintaining transactions or providing a valid EntityManager to our code. But we have no EJB container here, we need to do some things ourselves. But I still like to stick as close as I can to the "EJB" way, so I provide here my solution to the problem: the Dba class. It is really quite simple:


package com.yourapp.service;

import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;
import javax.persistence.Persistence;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/** some ugly code to make transaction management nice and dumb.
 *
 */
public class Dba {

  private static volatile boolean initialized = false;
  private static Boolean lock = new Boolean(true);
  private static EntityManagerFactory emf = null;
  
  protected Logger       logger      = LoggerFactory.getLogger(getClass());
  
  private EntityManager outer;
  
  /**
   * open Dba and also start a transaction
   */
  public Dba() {
    this(false);
  }

  /**
   * open Dba; if readonly no JPA transaction is actually started, meaning you will have no persistence store. You can still persist stuff, but the entities won't become managed.
   */
  public Dba(boolean readOnly) {

    initialize();
    openEm(readOnly);
  }

  public void openEm(boolean readOnly) {
    if (outer != null) {
      return;
    }

    outer = emf.createEntityManager();

    if (readOnly == false) {
      outer.getTransaction().begin();
    }
  }

  /** Get the outer transaction; an active transaction must already exist
      for this to succeed.
  */
  public EntityManager getActiveEm(){
    if(outer == null){
      throw new IllegalStateException("No transaction was active!");
    }
    
    return outer;
  }

  
  /** Close the entity manager, properly committing or rolling back a transaction if one is still active.
  */
  public void closeEm(){
    if(outer == null){
      return;
    }

    try{
      if(outer.getTransaction().isActive()){
        
        if(outer.getTransaction().getRollbackOnly()){
          outer.getTransaction().rollback();
        } else {
          outer.getTransaction().commit();
        }
      }
      
    } finally {
      outer.close();
      outer = null;
    }
  }


  /** Mark the transaction as rollback only, if there is an active transaction to begin with. 
   */
  public void markRollback(){
    
    if(outer != null){
      outer.getTransaction().setRollbackOnly();
    }
  }
  
  public boolean isRollbackOnly(){
    return outer != null && outer.getTransaction().getRollbackOnly();
  }

  
  // thread safe way to initialize the entity manager factory.
  private void initialize(){
    
    if(initialized){
      return;
    }

    synchronized(lock){
      
      if(initialized){
        return;
      }
      
      initialized = true;
      
      try{
        emf = Persistence.createEntityManagerFactory("testapp");
        
      } catch(Throwable t){
        logger.error("Failed to setup persistence unit!", t);
        return;
      }
    }
  }
}


This incredibly simple class takes care of two basic things:

- using one time on-demand initialization, it sets up the EntityManagerFactory
- it provides a way to create, obtain and properly close an EntityManager
- the Dba class provides an explicit "readOnly" mode which you should use when you are not going to persist any changes (effectively no JPA transaction is created in readonly mode)
- it provides a way for you to trigger a rollback

The ugly init code makes it pretty much self-sufficient, no need to trigger the setup using something like a context listener or whatever.

The usage of the class is really simple; you simply create an instance and from that moment on you use that one instance to manage a transaction. You may need to use the same transaction in multiple methods (DAO, service, whatever); I manage that by simply passing along the Dba object to the service methods. Careful usage of openEm()/closeEm() and getActiveEm() can prevent mistakes here. Example:


// high level service method, or perhaps even code in a JSF backing bean
public class doThingsAndStuff(){

  // open transaction  
  Dba dba = new Dba();
  try{
    // createUser 'adopts' the transaction
    UserDao.createUser(dba, "Someone", "some1", "Some1");
  } finally {
    // 100% sure that the transaction and entity manager will be closed
    dba.closeEm();
  }
}



public class UserDao {
  
  // low level DAO method in UserDao, expecting a transaction to already exist
  public static User createUser(Dba dba, String name, String username, String password){

    EntityManager em = dba.getActiveEm();
    User user = new User(name, username, password);
    em.persist(user);

    return user;
  }
}


The Dba class basically gives you the power that Bean Managed Transactions would give you when using EJB technology. You can start and close transactions at your leasure. To make a method adopt a running transaction, simply pass along the Dba object that is managing the transaction and be sure to use getActiveEm() to be 100% positive that the entity manager/transaction already existed. This is important because this way you can make dumb assumptions in your code; createUser() assumes that some other piece of code is responsible for closing the entity manager part of the Dba object that is passed to it. Now say that you make a mistake and you don't actually start a transaction before calling createUser(); then getActiveEm() will simply fail. Easy peasy.

Want to trigger a rollback? As you can see the closeEm() method provides for that; the responsibility you have is to mark the transaction as rollback only through the appropriate Dba method. The easiest way to do that is at the point where you deal with your exceptions. If that is "above" the layer of code that created the Dba object, you'll have to catch the exception, mark the transaction as rollback only and then rethrow the exception.

To minimize the strain on resources, you should only open a EntityManager transaction when you actually need one; which means you will be making changes that need to be persisted, or rolled back. When you only fetch data from the database, you don't need a transaction and thus you should not open one. Dba provides for this by allowing you to open it in readOnly mode.


public List<User> getAllUsers(){
  Dba dba = new Dba(true); // open in readOnly mode

  try{
    return dba.getActiveEm().createQuery("from User u order by u.name").getResultList();
  } finally {
    dba.closeEm();
  }
}


Almost cannot be made more lazy IMO :) You could change the design of Dba to default to readonly in the no-arg constructor; I leave that up to your own discretion.

If all that seems too complicated to you (or perhaps makes your eyes bleed), I'm sure you can come up with an alternative design that works for you personally. You have to find the balance between flexibility, robustness (which also includes catching mistakes you make) and ease of development. Ease of development is not only about minimizing code - it is also about maximizing the opportunity to catch mistakes. But before you pass judgement, I do ask you that you glance over the adding utility methods to Dba paragraph first.

This is by no means the only possibility you have of working with an EntityManager object in a web environment (for example there are inversion of control / bean injection frameworks such as Spring or Google Guice). But why bother when you can do it with a basic (stateless) application design and simply passing along an object?

Simple = better.

Then you get the discussion of how you would manage controller classes. Take the above example; you can distill from the code that UserDao.createUser is a static method. From your teachers to books to your colleagues, it will probably have been hammered into your head that you should avoid static methods and variables where you can. And those people and books are right, you should.

But this type of code you want to keep as stateless as possible, because stateless code is by definition thread safe, works in load balanced environments and there is no avoiding multithreading when you are doing a web application; there are going to be multiple users hammering on buttons at the same time. So I pose the question: why NOT use static methods in your controller classes? You could create an instance of the controller and then call its methods, but you could simply make the controller methods static and not be bothered with having to create an instance and basically be guaranteed that you are not going to slip in code that keeps state.

It is a very personal thing. I certainly don't mind creating static controllers to save me time and effort (and possible the creation of many short-lived objects). When you look at frameworks such as Play framework, it follows exactly the same design.

Simple = better.


JPA 2 - Beware of nested transactions

Using the Dba class it is quite easy to create multiple entity managers and have transactions running in parallel on the same thread. This is however not as fail-proof as it may seem as you may run into the issue of deadlock. The database you are connecting to, even an in-memory one like HSQL or H2, will likely employ table and/or row locking. When firing off multiple nested transactions you may inadvertently try to persist stuff to the same table at the same time, which may just result in a table lock wreaking havoc.

I ran into this puppy myself when using HSQLDB - the thread just hangs when you trigger a table lock. By switching in H2 I finally figured out what was going on (because H2 sets a lock timeout by default).

My remedy: don't use nested transactions. Just don't do it. You want to simplify your code layer; the transaction management should follow suit. Sure open and close the transaction as you see fit, but don't go making it any more difficult than it should be; try to keep one transaction active to one thread. Otherwise you will have to start to deal with the complexities of such an environment.

To maintain this, follow another simple rule: don't start a transaction in your backing bean. Only start (and cleanup) transactions in your controller methods. From there you can easily propagate the entity manager as I've already demonstrated; by simply passing it along as a parameter.


JPA 2 - Adding utility functions to Dba

I find the EntityManager interface a bit... verbose. There is more typing going on than is really necessary. Luckily with our Dba class as the base, we can add some wrapper code around EntityManager and Query to seriously reduce the typing involved.

Lets start with Query, which we will wrap in a class called QWrap:


public class QWrap {

  private Query q;
  
  public QWrap(Query q){
    this.q = q;
  }
  
  public QWrap par(String pname, Object v){
    q.setParameter(pname, v);
    return this;
  }
  
  @SuppressWarnings("unchecked")
  public <T> T single(Class<T> clazz){
    try{
      return (T) q.getSingleResult();
    } catch(NoResultException nre){
      return null;
    }
  }
  
  @SuppressWarnings("unchecked")
  public <T> List<T> multi(Class<T> clazz){
    return (List<T>) q.getResultList();
  }
}


Nothing too shocking; basically these methods are just short-hand versions of the Query methods you will probably use often. You can apply the same trick to other methods you tend to use. Now we can plug that into the Dba class:


public QWrap query(String q){
    
    if(outer == null){
      throw new IllegalStateException("Creating a query when there is no active transaction!");
    }
    
    return new QWrap(outer.createQuery(q));
  }


With a simple call to query() we will now obtain our QWrap object. Lets see it in action:


Dba dba = new Dba(true);

  try{
    List<Person> people = dba.query("select p from Person p where p.address.city=:city").par("city", "Washington").multi(Person.class);
  } finally {
   dba.closeEm();
  }


Neat huh? Note the abuse of generics here to basically get around having to do typecasts. But thats not all we can do. With a few more utility methods we can save even more typing. Lets begin by laying down a convention: all our entities shall have an auto incrementing ID column (using whatever strategy your DBMS provides). Using that, we can slap an interface on our entities:


public interface IdAcc {
    public long getId();
  }


This little interface is going to save us plenty of boilerplate code. You simply apply it to all your entities. With this baby in our toolbox, we can make a save() method that does a persist() or a merge() when required and returns the managed instance (if not in readonly mode):


public <T extends IdAcc> T save(T obj){
    
    if(outer == null){
      throw new IllegalStateException("Creating a query when there is no active transaction!");
    }

    if(obj.getId() > 0L){
    
      return outer.merge(obj);
    
    } else {
      outer.persist(obj);
      return obj;
    }
  }



I tend to avoid merge() myself if I can help it, but especially in a web application it can sometimes save plenty of time to just have JSF dump the changed values into an entity object and then merge it. In any case using the save() method you cannot only do that, you can easily create forms which can be reused for adding and updating data. More! Lets define a method to fetch an entity by ID and one to 'refresh' an entity, which is the act of making a detached entity attached again:


 public <T> T getById(Class<T> clz, long id){
    
    if(outer == null){
      throw new IllegalStateException("No transaction was active!");
    }
    
    return outer.find(clz, id);
  }
  
  
  @SuppressWarnings("unchecked")
  public <T extends IdAcc> T refresh(T obj){
    
    if(outer == null){
      throw new IllegalStateException("No transaction was active!");
    }
    
    return (T) outer.find(obj.getClass(), obj.getId());
  }


The getById() method isn't too special; its just a thin wrapper around the find method of the EntityManager. The refresh() method is incredibly useful however as you can just pass the entity object to it, and the method does the rest. The typecast in refresh() is necessary because of the added IdAcc interface. Not that you'll notice it is there in your calling code.


  User user = dba.getById(User.class, userId);
  user = dba.refresh(user);


This is just a little bag of tricks; I'm sure based on the ones I've provided you can add quite a deal more.


JPA 2 - Bootstrapping database data

I already discussed how to bootstrap a prototype web application's database here; I would give that a read if you want more information. For now, lets create a simple class in which we can insert initial data:


public class AppConfig {

  private static volatile boolean initialized = false;
  private static Boolean lock = new Boolean(true);

  protected Logger       logger      = LoggerFactory.getLogger(getClass());

  @SuppressWarnings("unchecked")
  public static List<User> getAllUsers(Dba dba){
    load();
    return dba.getActiveEm().createQuery("select u from User u order by u.name").getResultList();
  }

  private static void load(){
    
    if(initialized){
      return;
    }

    synchronized(lock){
      if(initialized){
        return;
      }
      
      initialized = true;
      
      Dba dba = new Dba();
      try{
        EntityManager em = dba.getEm();
        
        // insert all test data here
        em.persist(new User("testuser", "test1", "Test1"));
         
        // more init here
        
        logger.info("AppConfig initialized succesfully.");

      } catch(Throwable t){
        logger.error("Failed to setup persistence unit!", t);
      } finally {
        dba.closeEm();
      }
    }
  } 
}


Of course this code is only necessary when you use an in-memory database for prototyping or testing purposes; if you use a physical database then you don't need the AppConfig class at all.

The load() method, much like the Dba class, does an on-demand one time initialization. Reloading the webapp may cause this to happen a second time though, so you might want to build in a check if data already exists in the database to prevent data from inserted multiple times.

In this prototyping setup, it is your job to make sure that load() is called when it is required. This should happen automatically when you make the AppConfig class responsible for providing important data. User objects are a prime example, basically one of the first things you'll do is login most likely so that will trigger the database to be initialized very early on. You could also trigger it to happen in a context listener; then you don't need to have code in place that ensures it happens.

Note that when load() is invoked for the first time, that will be the moment where JPA is setup and the database is generated. So you won't know if you setup all that stuff correctly until you actually trigger your application to need the database.


JPA 2 - Getting something to deploy

So before we get to deploying the app and seeing if it works, we need some minimal stuff to make things happen. We at least need an entity to see that the database is being built, but we also need some bootstrapping code to get the database logic to initialize.

Let me copy a slimmed down version of the User entity from my prototype article:


@Entity  
public class User implements Serializable {  
  
  @Id  
  @GeneratedValue  
  private long id;  
    
  private String name;  
  private String username;  
  private String password;  
    
    
  public User(){  
  }  
    
  public User(String name, String username, String password, ){  
    this.name = name;  
    this.username = username;  
    this.password = password;  
  }  
  
  // getters and setters here...
}  


You can put it in any package you want, the class will be found automatically as soon as we cause the EntityManagerFactory to be created.

Now to get the Dba class to initialize, we need to use it at least once. Lets add an init method to our JSF backing bean to see that happening quickly:

@ManagedBean
@ViewScoped
public class Index {

  public Index(){
  }
  
  @PostConstruct
  public void init(){
    // breaking my own rule: starting a transaction in the backing bean. This is only for demonstration purposes however.
    Dba dba = new Dba();
    try{
      // some stupid code that triggers the database to be created
      AppConfig.getAllUsers(dba);
    } finally {
      // cleanup the transaction
      dba.closeEm();
    }
  }

  public String getHello(){
    return "hello";
  }
}


The PostConstruct method is abused here to bootstrap our database, just so that you can see it happening. Like I stated before you don't want to ever see the Dba class being used in a backing bean so some sort of Controller method would normally be called here. But I don't want to litter the article with too many pieces of fake code.

As soon as you deploy the application and start the server, nothing special will happen. But as soon as you navigate to the application's index page, the call to Dba will be triggered which will in turn cause the database to be initialized (you'll notice how tables will be created in the logging) and the data bootstrapping code to be executed.

And that's basically it - setting up a project for JSF 2 and JPA 2 to deploy on Tomcat 7, using Eclipse as the IDE. At this point you'll be ready to write some code. The remainder of this article will deal with annoyances and troubles you may run into from this moment on.


Dealing with multiple environments (dev, acceptance, production)

When you develop an application, at one point you will want to publish it to somewhere that is not your own development computer; be it a testing environment or live production. The tricky part about that is that each environment will require certain settings to be entirely different; most notably the datasource will need to point to a completely different database. Fortunately because we use Tomcat, that is relatively easy to manage and Eclipse has already helped us a bunch too; what you do is utilize the context.xml file. We have ours in our development environment neatly tucked away into that Servers project. This file is not part of our project and so will never ever be deployed with the application to any other server.

But as said in the previous section, we don't want to put application specific configuration in the global context.xml file or else we'll have difficulties deploying multiple applications at the same time. Instead tomcat allows us to specify a specific configuration file per-application. You put this file in a slightly difficult to formulate directory inside the tomcat_dir/conf directory; the format is:
tomcat_dir/conf/Catalina/HOSTNAME/WARNAME.xml
So when deploying on the localhost and the war name is testapp.war, the directory and file would be:
tomcat_dir/conf/Catalina/localhost/testapp.xml
Inside this file we can then put the information we put in the context.xml in our development setup:

<Context>
  <Resource bla bla bla/>
</Context>

You can try to do this in your development environment, but you'll find that Eclipse will throw away your file again. As I said - it has taken control over your Tomcat installation. Hence my suggestion to put it in the global context.xml file instead.

Now for a bit of a fun addition, what if we want to know from within the application code if we are running in a development environment or not? That might sound like a design smell, but I for one tend to allow certain things in a development environment that I do not allow in any other environment to expedite the development process; think of relaxing security measures and such. We can facilitate something like that quite easily by defining a property in the JNDI context which we can interrogate from the code. In the global context.xml file (that file in the Servers project) you can add this inside the context element:

    <Environment name="DEPLOYMENT" value="DEV" type="java.lang.String" override="false"/>

And then in the code we can do this:

  Context initCtx = new InitialContext();
  String env = (String) initCtx.lookup("java:comp/env/DEPLOYMENT");
  boolean dev = "DEV".equals(env);

On your test/acceptance/production server you then simply do NOT define this element and the boolean variable will end up as false. Easy!

You could add any properties you want of course; if you want them to be global add them to the context.xml, if you want them to be local to one application only, then add them to the application specific xml file.


Tomcat 7: Dealing with republishing

The Java debugger has a neat hotswap function where code changes can be "uploaded" to a running JVM instance, effectively allowing you to change code of a webapp as you have it open in your browser. Since this is a feature of the JVM debugger, it is only actually activated when you start the server in debug mode from Eclipse. This works up to a certain point of course; you cannot make changes to the class structure for example without a full webapp reload being needed.

However when you try to make a change to a class or resource while deploying to Tomcat, you may run into the problem that the entire web application reloads (but only if you did not follow my instructions so far)! Not only is that highly annoying and time consuming, but it will quickly make you run into the dreaded "out of permgen space" problem (for which there may be some steps to attack it; check the comments of this article). This problem was especially apparent for me as I bundled Richfaces when starting out with this article; two redeploys is all it took before I had to reboot the server.

Fortunately, this has a very easy fix; by simply telling Tomcat it should not automatically reload the application. This is that "reloadable" property I told you to change all the way back in the section titled fixing the deployment settings. At this point if you still had it on simply disable it, republish the server and try again; you'll find that the webapp no longer redeploys, but the changes you make, even changes to for example a JSF backing bean method, are still automatically visible. Don't forget to hit reload in the browser of course ;)


Eclipse: Web resources not hot-deploying?

I ran into the issue where using this hot-deployment scheme, classes were being properly redeployed but web resources in for example src/main/webapp or src/main/resources were not, even though those directories was part of the deployment assembly! After struggling with it for a long time I figured out that I screwed up the hot-deployment settings; I had set the server publishing settings to never publish automatically; by default that should be automatically publish when resources change. Check that Eclipse is allowed to publish. When you make a change, don't forget to set the reloadable property of the web application to false as described above.


Final thoughts: adapt to your needs

Of course JSF is only an example; you can substitute any web framework you want. You could yank out Primefaces and use vanilla JSF or another addon framework (Icefaces, Richfaces). What I wanted to do was create an environment that is very close to an enterprise application but without all the added weight to it. You don't need an application server, you don't need an EJB layer, you don't need an application built out of multiple tiers, layers, boilerplate code and other madness that generally makes Ruby or PHP scripters snigger at you behind your back.

Simple = better. Also with Java as the base it can be simple... with some major effort to get it to that point. Always those dreaded configuration files, we don't seem to be able to ever be rid of them no matter how hard people try. I hope this article can help you wade through the boring setup stuff and get you to the juicy meaty side of web development; actually creating something and seeing it appear in your browser!