Asynchronous job triggering in Grails 2.0.x

There are times during application development where we need to accept request to do some data processing which is going to take a long time. This cannot be completely implemented in the traditional request/response paradigm.

The idea is to use the traditional REST endpoints and accept the request, delegate it to a processing engine, and send the response right away. Consider this web method in a grails controller, which accepts a search query and has to mine the web to obtain the result.

  def boilTheOcean((@RequestParameter('query') final String someSearchStr) {
	  def root = null
	  def builder = new groovy.json.JsonBuilder()
	  def miner = new MinerService ()
	  miner.mineTheWeb(someSearchStr)

	  root = builder.messages {
  	     message   "Request delegated for processing" 
	     status    true
	  }
	  render(text: builder.toString(), contentType: "application/json", encoding: "UTF-8")
   }

The above code instantiates the MinerService and delegates the task of mining the web for the given search string.

Async processing:
There is an excellent library GPars (Groovy Parallel System) http://gpars.codehaus.org/ that has a few asynchronous processing models that can be used (Fork/Join abstraction, Dataflow concurrency, Actors etc…). With the exception of some groovy scripts, I have used it in every single Groovy project – I cannot recommend it enough!

For this example, we are going to use Actors (GPars examples. For more information on Actors, its history, theory & model – read this Wikipedia entry (Actors)

Create an Actor to process the request asynchronously. The onMessage is the handler for request coming into the Actor and this is invoked asynchronously.

import groovyx.gpars.*
import groovyx.gpars.actor.DynamicDispatchActor

final class WebMinerActor extends DynamicDispatchActor {
	
    static final Logger log = Logger.getLogger(this)
    void onMessage(String searchString) {
	 new WebMinerAggregatorProcessor().mineWWW(searchString)
	 log.debug 'Completed mining!'
    }
}

The mineTheWeb() method on MinerService that is invoked from the Grails controller

	def mineTheWeb(String searchString) {
    	    WebMinerActor actor = new WebMinerActor ()
	    actor.start()
	    actor.send(searchString)
	    log.debug("sent to actor to mine for: ${searchString}")
 	    
            //Express return, does not wait for Actor to finish processing.
	    return 
	}

It is important that Actors have no shared state. When start() is called, the Actor is started and a START_MESSAGE is sent to execute any afterStart handlers. The send() sends the information to the actor, where the appropriate onMessage() method is invoked. At this time, you would see the debug message displayed on the console and the method returns to the controller.The controller constructs a JSON response and sends it to the client. The debug statement “Completed mining” will be printed a long time later (depending on the time taken to execute the operation).

This is just an illustration, if you are looking for best practices…

  • The service should generate a token for the request and should return the token to the controller, which should return it to the client. The subsequent follow up requests from the client should use the token to trace the processing status or result.
  • The other option is to push the result to the client, if you have the ability to do so. There are situations where this might be preferable to polling.
  • For really long running requests Actors may not fit the bill, especially if you want to mine the web, if you want to have your processing done on another host, another VM, you can use an Service Bus model, where there is a request processing chain, that can queue, delegate processing to other nodes and aggregate results.
  • Use a Service Bus to delegate it to a Hadoop process to do the mining work.

Notes:

GPars ships with Groovy 1.8.6 and above. If you happen to use a different Groovy version, manually add the jar to your Grails lib directory.

There is a Grails plugin called Background Service, which does the same task, but in the Grails way ( link – as in managed plugin, auto controller injection) . I did not have much luck with it, hence I decided to go with the GPars approach.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: