Monday, July 9, 2012

JAX-RS client: beyond hello world

So, RESTful webservices. You came here, so you must be interested in invoking a RESTful webservice from a Java client. At this point you must have also figured out that JAX-RS is in fact the way to go when it comes to invoking RESTful webservices, even though technically you could do it with a simple URLConnection or a little more robust: the Apache HttpClient API. Of course JAX-RS offers some benefits, the biggest one being that it takes a big pile of work away from you.

Before you can use JAX-RS, you must first understand JAX-RS. The primary thing to understand is that it is only a specification: it doesn't actually do anything. You can't download JAX-RS and expect to be making connections to servers. To actually be able to do anything you need something that implements the JAX-RS specification and in true Java fashion, you have a plethora of choices. Third party offerings include CXF, Restlet and RESTEasy - Oracle delivers a reference implementation called Jersey; that's what I'll be using for the purpose of this article but since its a specification, it doesn't really matter which implementation you're using; the difference will mostly be in the setup part. If you deploy the client as part of a JEE 6+ application, a JAX-RS implementation is already provided by the container.

Also be aware that Jersey is already an API with a development history; it has grown up a lot since its 1.0 days with plenty of added features and improvements in the way things are done; some of which are not exactly documented clearly yet. Plenty of articles and blogs on the net will refer to how things were done in the earlier days; do keep an eye out for the date of a posted article to know how accurate the information in it is going to be. For this article I made an effort to figure out what is the more up to date way to use the API (at the time of writing of course), but perhaps I missed something.

The Jersey website actually hosts a somewhat useful reference manual; I say somewhat because like most reference manuals its a document without focus and therefore hard to follow in that "I'm new, how do I begin" state that we all begin in. So it didn't stop me from having to hunt through many a forum to get to the point where I could actually effectively put JAX-RS to work for me. Most articles I found on the net of course went no further than the bare basics; how do you setup a hello world service, how do you create a hello world client. Well not me, I'm diving a bit deeper here. To not make the article too complex I'm only focusing on creating a RESTful client here. The server/service part of it I have put in a dedicated article (because again: had to hunt through forums, make guesses and do plenty of experiments to get it to work the way I wanted).

What this article will cover is the following:
  • Maven setup / dependencies
  • how to connect to a server
  • how to invoke a service and get the results - with nested objects
  • how to deal with collections and arrays
  • how to post data to a service
  • performance enhancement
  • dealing with authentication

Then there is the question of protocol: what data format to use? The most common ones are XML, YAML and JSON - I'll be going for the one that makes the most sense to me, which is JSON.

Before we begin I'd like to stress one thing: I'm not a JAX-RS expert. This article is in no way in its final form and will continue to grow much like most of my articles expand and improve over time as I continue to learn from my own mistakes. Feel free to point out glaring mistakes or omissions in the comments; I actively maintain my articles so you can be quite sure that mistakes will be fixed.

Setting up Maven

To get going with Jersey you need only a few dependencies which are thankfully in Maven central. To keep it basic lets assume you want to deploy to Tomcat and not a full JEE application server. You would declare your dependencies as follows:

    <dependency>
      <groupId>com.sun.jersey</groupId>
      <artifactId>jersey-client</artifactId>
      <version>1.18.1</version>
    </dependency>
    <dependency>
      <groupId>com.sun.jersey</groupId>
      <artifactId>jersey-json</artifactId>
      <version>1.18.1</version>
    </dependency> 

To know the latest version of Jersey, simply consult Maven central search or here: Jersey in Maven central

If you are deploying to for example Glassfish 3 or JBoss 7 you would mark these dependencies as provided, since all required dependencies are already there in the container. You will probably want to figure out which JAX-RS implementation is delivered with your application server so you can make your compile time dependencies match up.

That's it for the client (to also provide a RESTful service, you would include the jersey-server artifact). Of course Maven being Maven, this will provide you with a whole host of transitive dependencies. For the people who don't use Maven (please, come to the dark side) the following list of dependencies are actually going into the webapp with the Jersey 1.18 version:
  • jersey-core 1.18.1
  • jersey-client 1.18.1
  • jersey-json 1.18.1
  • jettison 1.1
  • activation 1.1
  • stax-api 1.0.2
  • jaxb-api 2.2.2
  • jaxb-impl 2.2.3-1
  • jackson-core-asl 1.9.2
  • jackson-jaxrs 1.9.2
  • jackson-mapper 1.9.2
  • jackson-xc 1.9.2
Note that both JAXB and Jackson are in this list. Both of these APIs are used by Jersey to do mapping of data to objects; JAXB for XML and Jackson for JSON. For ease of deployment JAXB annotations can be used on your pojos to provide the mapping information no matter which data format is used; Jersey will still use Jackson under the hood to do the actual mapping work.

What about a server?

To be able to build a client you need a server that can be invoked, at least to test stuff out. Unfortunately that is a bit of a challenge posed to me in this here article. But in order to understand how the client works you don't really need a server; you only need what the server outputs which in our case is a blob of JSON formatted data. So without further ado lets define that exact blob.

Somewhere in the world we have a service that allows us to search for projects. A project has basic properties such as a name and a deadline, but it is also tied to a company. A company on its own has a name and an address and a list of employees. Sounds pretty basic, but in that one example we cover nested objects and a collection.

In JSON terms, that would translate to the following which I'll pretty print for the purpose of making it possible to see what's going on (in a normal circumstance you don't want unnecessary whitespace in your datastream):

[{
  "active":true,
  "id":1,
  "name":"Razorblade supershave mark 2",
  "company":{
             "id":1,
             "name":"Razorblade Ent.",
             "employees":[{"id":1,"name":"harry"},{"id":2,"name":"sally"}]
            }
 },
 {
   "active":true,
   "id":9,
   "name":"fortune cookie dispenser",
   "company":{
              "id":2,
              "name":"Unfortunate Inc.",
              "employees":[{"id":5,"name":"gordon f."},{"id":7,"name":"sam stone"}]
             }
 },
 { "active":true,
   "id":15,
   "name":"fortune cookie baking device",
   "company":{
              "id":2,
              "name":"Unfortunate Inc.",
              "employees":[{"id":5,"name":"gordon f."},{"id":7,"name":"sam stone"}]
             }
 }]

Consider this a full dump of the database; there are only three projects in here. This is not typed by hand by the way - this is the output of a Ruby On Rails service that I created for testing purposes. The fact that I could whip up my Java client and it "just worked" is what really made me go "WOW!", that hasn't happened a very long time. In my JAX-RS server article I'll be basically creating a simplified Java version of the service that produces the exact same output.

If you're not familiar with JSON I urge you to go read up on it a bit, but the above should still be human readable for a programmer. [] represents an array of elements, {} represents the start of a complex type which in Java terms will translate into a POJO bean. Quite easy to read, not as easy to type.

Defining the model

I mentioned POJO beans. To be able to read out the above JSON data we'll need a model to be able to store the information. Three complex types makes three Java POJO classes. I'll leave out the getters and setters to save some space.

@XmlRootElement
public class Project {

  private boolean active;
  private long id;
  private String name;

  private Company company;

  // getters and setters here
}


@XmlRootElement
public class Company {
  private long id;
  private String name;
  private Employee[] employees;

  // getters and setters
}

@XmlRootElement
public class Employee {
  private long id;
  private String name;

  // getters and setters
}

Only a few things to note:
  • the XmlRootElement annotation; that is a JAXB annotation that basically marks the pojo class as a complex type to be used in mapping the data; either JSON or XML.
  • the properties of the beans match the labels as you'll find them in the JSON data
  • array of employees in the JSON data, array of employees in the Java class. You can however replace the array type with a List<Employee> type if you so please, it magically works.
Convention over configuration: you're best off just using the exact same property field names as you find them in the JSON data, or else you'll have no choice but to add extra annotations to provide mapping names. With our model in place we can now go ahead and write our surprisingly simple client call.

Invoking the call

Now we get to the interesting part. The webservice we're talking with provides a service which allows us to search for projects by a (partial) name. The imaginary RESTful url to invoke is:

http://www.superpeople.com/REST/

Under that URL are all the different services, of which our project search which can be invoked as follows through a GET request:

http://www.superpeople.com/REST/project/search/:NAME

In that url :NAME is to be replaced with our partial project name. The services is structured such that all project related functions are under the /REST/project/ url. Lets define the simplest way to invoke this service and get the results without worrying too much about performance just yet. This is done in only three steps:

1) obtain a JAX-RS Client instance
2) create a WebResource using this Client to the URL you want to invoke
3) actually invoke it, optionally setting some parameters

Creating a Client is very little work and is pretty much boilerplate code. There are multiple ways to actually create a client, I picked this as the "way I'll always do it":

  public Client createClient(){
    
    ClientConfig clientConfig = new DefaultClientConfig();
    clientConfig.getFeatures().put(JSONConfiguration.FEATURE_POJO_MAPPING, Boolean.TRUE);
    clientConfig.getProperties().put(ClientConfig.PROPERTY_FOLLOW_REDIRECTS, true);    
          
    Client client = Client.create(clientConfig);
    return client;
  }

The ClientConfig class allows for some basic properties to be set; most of which you will likely not ever have to deal with. The important one here is the FEATURE_POJO_MAPPING setting: this tells the JAX-RS client that we'll be dealing with JSON data.

Okay on to the actual web invocation.

 
public static final String HOST = "http://www.superpeople.com/REST/"; 
public static final String PROJECT_URL = HOST + "project/";
public static final String PROJECT_SEARCH = "search";

public Project[] searchProjects(String name){
  
  Client client = createClient();
  WebResource res = client.resource(PROJECT_URL);

  return res.path(PROJECT_SEARCH + "/" + name)
            .accept(MediaType.APPLICATION_JSON_TYPE)
            .get(Project[].class);
}

The first thing that may seem off to you are those constants which are hardcoded; later on we'll make that a little more robust and a little less hardcoded but for now lets keep it simple. Basically we split the resource up into two parts: the 'project' resource and the 'search' function. As you can see a WebResource is constructed for the project resource. Later on we'll see why this is good design, but for now take good notice of that path() method being called; there we pass in the actual function that we want to invoke plus any variable parameters that the service call may provide. In our example the search demands a partial project name to be passed, so that's what we dump into the path.

Let's say that we search for 'fortune'; the full URL invoked will then become:

http://www.superpeople.com/REST/project/search/fortune

The imaginary service will then do a search for projects with a name starting with the word 'fortune';which in the case of our massive three project database should return two results.

Moving along to the accept method, that shouldn't need much imaginative power. Here we declare that the service should expect JSON data.

Finally the method that does all the work: get. Upon this call the restful service is actually invoked and the results will be translated into that which we pass into get() as a parameter; in this case an array of Project objects. And that's where all you have seen so far comes together:

- our JSON data structure as delivered by the phantom web service
- the pojo classes with the minimal annotations that matches said JSON data structure
- the JSON support we activated through the ClientConfig settings

All that equals magic happening during the get() invocation (because this is a GET request; if it would be a POST request you would use the post() method). Jersey will at this point read the JSON data and under water use the Jackson API to translate that data into the object hierarchy we've built, including mapping to child objects and even our employee array in the Company object; it just works. And if it doesn't work then you'll get a very clear error stating where the typo is.

Of course even though magic is being performed, its still magic under your control. YOU have to define the proper pojo object structure with proper naming. YOU have to know the exact URL layout of the service to invoke. YOU have to know in what way data will be returned (single object, array of primitives, array of objects). The tool is not doing the work for you - it is making your work easier. Exactly how it should be.

Now we've built our client call to return an array of projects. But what if you wanted a list of projects in stead? Simply swapping out the array type with a List type doesn't cut it in this case, this is how you do that:

  return res.path(PROJECT_SEARCH + "/" + name)
            .accept(MediaType.APPLICATION_JSON_TYPE)
            .get(new GenericType<List<Project>>() {});


The GenericType object is there to help the translation to the proper generic List type in such a way that the compiler likes it. From now on, it's copy/paste logic.

URL parameters

Let's say the service definition is actually that you invoke the URL like this:

http://www.superpeople.com/REST/project/search?name=fortune

The only thing that changes is the way we do the invocation.

  return res.path(PROJECT_SEARCH)
            .accept(MediaType.APPLICATION_JSON_TYPE)
            .queryParam("name", "value")
            .get(Project[].class);

If you have multiple query parameters then simply call queryParam() multiple times. Alternatively you could use this too:

  MultivaluedMap<String, String> params = MultivaluedMapImpl();
  params.add("name", name);

  return res.path(PROJECT_SEARCH)
            .accept(MediaType.APPLICATION_JSON_TYPE)
            .queryParams(params)
            .get(Project[].class);

Which would allow you to set multiple parameters in one go, at the expense of more boring boilerplate code.

Time to put that to the test

You're now already at the point where you can try it out. Since you are interested in creating a JAX-RS client, you must have some sort of RESTful service that you want to be able to invoke. If you do not - well you can always try to create one for example by following my server article.

But of course where there is a will to be lazy, there is a way to be lazy. You don't need any service; the only thing you need is a way to do a GET request to some text file which will literally return the JSON data that I provided earlier. Paste that in a file test.json and make sure you get get to it through a web url even if it is on the localhost. Then use the following bit of code to use it:

  WebResource res = client.resource("http://localhost:8080/test.json");

  return res.accept(MediaType.APPLICATION_JSON_TYPE)
            .get(Project[].class);

It should work just fine even in the pretty printed format.

The reverse situation: sending data

Until now we've been leeching information from our phantom service. What if we'd actually want to return something? Like create a new company to which projects and employees can be bound?

That is in fact quite simple but it requires a little theory. Until now we've been dealing only with GET requests; in other words requests that fetch something. To do modifications we need to use the other HTTP request methods: POST, PUT and DELETE. RESTful webservices tend to define a convention: POST is used to add information, PUT is used to update information. I hope I don't have to explain what delete will be used for :)

So in order for our client to be able to send something we first need the contract; our phantom service requires us to POST JSON data to the service url /project/company/add and will in return give a JSON complex type as an answer in the form of this IdResult object:
@XmlRootElement
public class IdResult {

  private long id;
  private boolean ok;
  private String msg;
  
  public IdResult(){
  }
  
  // getters and setters here
}

Which in JSON terms may look something like this for a success result:
{"id":"1341924925250","ok":"true"}

With that knowledge and the IdResult object at hand, we can construct our client call.
public static final String COMPANY_ADD = "company/add";

public IdResult addCompany(String name){
  
  Client client = createClient();
  WebResource res = client.resource(PROJECT_URL);

  Company company = new Company();
  company.setName(name);

  return res.path(COMPANY_ADD)
            .accept(MediaType.APPLICATION_JSON_TYPE)
            .post(IdResult.class, company);
}

And really, that's all there is to it. JAX-RS takes care of translating our Company object to the JSON body of the POST request and will also translate the JSON response sent back by the service into the IdResult object through that one post() invocation. Powerful stuff huh?

Making it better: stop hardcoding stuff

In the code I gave you until now the actual host of the service was embedded in the code. That may work for development and testing purposes, but if you put a REST client in production you will want to be able to manage the actual host URL outside of the code; for example you may have a completely different host URL for test/acceptance and production environments, in which case it depends on where the application is deployed which URL is to be used.

How to actually configure the URL depends on what you have available. Some options:
  • Pass it as a system property to the java command (-Dresturl=...)
  • In case of JBoss: define it in the property service
  • Store it in a settings database table
There are probably more solutions to be found but I believe in the power of no more than three and lets be honest: the first impulse anyone is going to have is store it in a database. My JSF 2 and JPA 2 on Tomcat 7 article can help you to write the code to do exactly that.

Making it better: re-use objects

The code as it is now has you creating a Client and a WebResource each time you do a call. These are quite expensive operations and as it turns out: you need to create them only once. We're talking about a stateless design here, you can safely re-use both the Client and the WebResource class in multiple threads without risk of collision. So lets do exactly that. First of all, lets isolate the actual initialization. How to design the code is pretty much a personal thing. I like to have setup in one place, so I tend to create an abstract base class in which I deal with it. Something like this:
public abstract class RestBase {

  public static final String PROJECT_URL = "/project/";
  
  protected static Client client = null;
  protected static WebResource PROJECT_RESOURCE = null;
  
  public static void init() {
    
     ClientConfig clientConfig = new DefaultClientConfig();
     clientConfig.getFeatures().put(JSONConfiguration.FEATURE_POJO_MAPPING, Boolean.TRUE);
     clientConfig.getProperties().put(ClientConfig.PROPERTY_FOLLOW_REDIRECTS, true);    
          
     client = Client.create(clientConfig);
     
     // whatever database logic you have to get the host name from the database
     String hostname = SettingsDao.getSetting("REST_host");

     PROJECT_RESOURCE = c.resource(hostname + PROJECT_URL);
     
     // initialize other resources here
  }
}

This is the base class for all RESTful client services I'll be creating (which just happens to copy/paste very nicely). It has a method to initialize the one Client object we'll be exposing On top of that it initializes all the RESTful resources in one go and exposes them to subclasses. What's with all the statics I hear you wonder!? I know, it's not object oriented design. But so what? This code at least is easy to use, as we'll see in a moment. Next up is the project client service:
public class RestProject extends RestBase {

  public static final String PROJECT_SEARCH   = "search";
  public static final String PROJECT_GET      = "get";

  public static List getProjects(String name) {

    return PROJECT_RESOURCE
            .path(PROJECT_SEARCH + "/" + name)
            .accept(MediaType.APPLICATION_JSON_TYPE)
            .get(new GenericType<List<Project>>() {});
  }

  public static Project getProject(long pid) {

    return PROJECT_RESOURCE
            .path(PROJECT_GET + "/" + pid)
            .accept(MediaType.APPLICATION_JSON_TYPE)
            .get(Project.class);
  }

Huuuuu, more statics. You may not like that, in which case I challenge you to change it to your liking. Me? I think 100% stateless design is only hindered by the need for object creation before you can actually invoke something.

Note how I added a second phantom call just to make a little more exciting. The JSON data that this call may return to match our client code will look like this:

{
  "active":true,
  "id":1,
  "name":"Razorblade supershave mark 2",
  "company":{
             "id":1,
             "name":"Razorblade Ent.",
             "employees":[{"id":1,"name":"harry"},{"id":2,"name":"sally"}]
            }
}

So only one "complex type" which maps to our Project pojo class. As you can see the code manages that by simply passing in Project.class to the get() method. You know, for once this is an API that is really cleverly designed.

Now you also see the benefit of splitting the project resource from the actual functions that are exposed through the project URL; the same 'project' web resource can then be re-used to call the different functions ('search' and 'get' in this case). That minimizes the amount of bootstrapping that needs to be done.

Speaking of bootstrapping: that init method still needs invoking. A context listener can help us do that. The listener will be invoked upon deployment of our web application.

@WebListener  
public class InitContextListener implements ServletContextListener {
  
  private static volatile boolean initialized = false;
  
  public void contextDestroyed(ServletContextEvent ev) {
    
  }

  public void contextInitialized(ServletContextEvent ev) {

    if(initialized == false){

      RestBase.init();
      initialized = true;
    }
  }
}

And there you have it. Now as I said most of this is subject to personal taste. You may in fact skip the RestBase class altogether and simply initialize a Client instance per client service class you create. You may move all the setup logic in RestBase directly into the InitContextListener. If performance is not much of an issue because you do only a handful of calls sporadically, you may just want to use the "simple" way of doing things that the article began with. Don't re-use anything, just initialize on the spot, use it and discard it.

Whatever you feel comfortable with, and I do urge you to think about it and experiment a bit with it to find that which is comfortable to you (and your colleagues).

You can also simply run as a command line application at this point, assuming you have your project dependencies properly setup (goes pretty much automatic when you have a Mavenized project).

public static void main(String[] args){
  
    RestBase.init();
    
    List<Project> projects = RestProject.getProjects("fortune");
    System.out.println("Got " + projects.size() + " projects!");  
}


Plugging in Apache HttpClient

By default Jersey will simply use the JDK features to do HTTP calls, which boils down to the usage of HttpURLConnection and HttpsURLConnection. For simple services this will be sufficient although people have been known to experience sporadic failures when using these classes in combination with certain (unstable?) web servers. No matter since these classes have been basically made obsolete by the incredibly powerful, feature complete, fast and easy to use HttpClient built by our friends at Apache. So is there a way we can make Jersey use it?

Yes we can, through the following setup code. First of all an additional dependency for our Maven pom:

<dependency>
    <groupId>com.sun.jersey.contribs</groupId>
    <artifactId>jersey-apache-client</artifactId>
    <version>1.18.1</version>
</dependency>

For the non-maven users, this adds the following additional dependencies to the project:
  • jersey-apache-client 1.18.1
  • commons-httpclient 3.1
  • commons-logging 1.0.4
  • commons-codec 1.2
With that dependency in place, you can construct the client differently like this:

Client c = ApacheHttpClient.create(clientConfig);

And then you've succesfully plugged in HttpClient in place of the less reliable JDK URLConnection APIs. At this point you may also want to activate automatic cookie handling support which is one of the more useful bonuses that HttpClient gives you:
  ClientConfig clientConfig = new DefaultClientConfig();
  clientConfig.getFeatures().put(JSONConfiguration.FEATURE_POJO_MAPPING, Boolean.TRUE);
  clientConfig.getProperties().put(ClientConfig.PROPERTY_FOLLOW_REDIRECTS, true);    
  clientConfig.getProperties().put(ApacheHttpClientConfig.PROPERTY_HANDLE_COOKIES, true); 
  
  Client c = ApacheHttpClient.create(clientConfig);

And that's basically all there is to it; try running whatever test code you have created and you'll see that things still work as they did before. In my experience: a bit faster even. The downside of using HttpClient is that you are of course weighing down your application with even more dependencies.

Dealing with HTTPS

One day this article will have a step-by-step guide on how to not only write the code but also setup a test environment. Until then I defer to this most excellent stackoverflow thread which should be the kick in the right direction that you need:

http://stackoverflow.com/questions/3434309/accessing-secure-restful-web-services-using-jersey-client

Authentication

Authentication can be solved at different levels. First of all there is the API level; the service itself may have some sort of authentication scheme built in, for example requiring you to pass along a HTTP header or certain parameters that identify who is calling. Sending a HTTP header is easy, like this:

    return PROJECT_RESOURCE
            .header("NAME", "VALUE")
            .path(PROJECT_SEARCH + "/" + name)
            .accept(MediaType.APPLICATION_JSON_TYPE)
            .get(new GenericType<List<Project>>() {});

The header() method does all the work. Invoke it multiple times to add more headers.

It may be that the call is actually secured based on IP filtering. Then there is nothing to do with or in code, the firewall of the service provider will either allow or disallow calls from your network. To gain access the firewall needs to be configured appropriately; good luck dealing with tech support.

It may also be that a form of HTTP authentication is employed. Contrary to what the online documentation will try to make you believe, setting up something like basic HTTP authentication through JAX-RS is really as simple as this:

client.addFilter(new HTTPBasicAuthFilter("username", "password"));

And more secure digest authentication:
client.addFilter(new HTTPDigestAuthFilter("username", "password"));

This works just fine with either the "default" Client or the Apache HttpClient.

Finally there is OAUTH, which is an authentication solution and API that has become somewhat of a standard on the net through popularity. Unfortunately I have no experience with using the API just yet so I cannot document it, but some initial scouting on the net seems to indicate that OAUTH support for Jersey is actually available.

Time to call it a day

There you have it. Less than an entire manual on JAX-RS, more than hello world. I hope in this one article I managed to capture the more common use cases for JAX-RS on the client side of things and you are now well on your way to actually being productive. At least I was after having pieced all this stuff together.

For further reading I suggest you get a decent book on the API, but you may also want to check out the JEE 6 tutorial chapter on it, which is likely going to be a slightly less chaotic approach to the material than the Jersey reference manual is but will focus mainly on the programming part. And let me plug my JAX-RS server article one more time :)

Good luck with your project!

No comments:

Post a Comment