Grails & Quartz – periodic job execution in Grails 2.0.x

There are times during web application development, where we would like to build a cron-based trigger to accomplish some background tasks. These could be – greedy cache fetchers, process to ftp files periodically, temporary database table cleaners, spawn bots to bring down the internet etc…

In Grails , there are 2 ways of doing it

  • Use Unix Cron to setup a job & timer
  • Use Quartz to trigger a job.

In this post – we will look at using Quartz.

What are we trying to solve:
Say we have processes in our application that generates temporary files and are not good enough to clean up after themselves. We can write a job that fires periodically and deletes those files on the different nodes that we have in our environment.

Since there are many hosts in our environments, we cannot fire of a job that connects to each node sequentially, we will use some amount of parallelism (GPars of course !)

We will use this contrived example to illustrate the use of the Quartz-Grails integration.

Installing the quartz plugin
On your Grails prompt

  install-plugin quartz2

This would download and install the Quartz2 plugin (link). Once the plugin has been successfully installed, we can now create a POGO that will hold the configuration information for the job.
I have used a domain model and marked it as a non persistable.

import groovy.transform.ToString

@ToString
class QuartzConfig {

	// Non persistable domain class
    static mapWith = "none"

	String jobName
	String jobGroupName
	Integer startsAfter
	Integer repeatInterval
	Integer numThreads
	List nodes
	String whichEnv
	
}
  • The jobName, jobGroupName are values to identify your job. This could be any text.
  • The startsAfter indicates after how many minutes after the server start would this job be triggered. Say if you restart the server and you do not want to run this right away but would like to wait for 15 mins before firing the process, you can use this attribute to set an appropriate value.
  • The repeatInterval indicates how often you would like to fire the job
  • The numThreads attribute identifies the number of parallel threads you plan to spawn to accomplish the task. This is not compulsory, but for the natire of job that I have parallelism would help!
  • The nodes contain the list of nodes that we have to access to clean the temp files.
  • The whichEnv indicates in which environment is this job running – dev/test/prod
  • Now we open the Config.groovy file (in the conf directory) and add the following entry

    
    QuartzConfig quartzConfig = null
    ...
    
    environments {
        development {
     		quartzConfig = new QuartzConfig(
                       jobName: "fileRemover" 
                      ,jobGroupName: "lumberjack"
    				  ,startsAfter: new Integer(1) 
    				  ,repeatInterval: new Integer(5)
    				  ,numThreads: new Integer(4)
    				  ,whichEnv: "development"
    				  ,nodes: ["node1.test", "node2.test","node3.test"]
    				)
        }
        production {
     		quartzConfig = new QuartzConfig(
    				   jobName: "fileRemover" 
    				  ,jobGroupName: "lumberjack"
    				  ,startsAfter: new Integer(15) 
    				  ,repeatInterval: new Integer(20)
    				  ,numThreads: new Integer(8)
    				  ,whichEnv: "production"
    				  ,nodes: ["node1.prod", "node2.prod","node3.prod","node4.prod","node5.prod"...]
    				)
     }
     ......
    }
    

    In production start the job 15 mins after the server starts and run the job every 20 mins, using 8 parallel threads. The nodes to work against are specified in the node

    At the bottom of the Config.groovy file – add this entry

    grails.plugin.quartz2.jobSetup.fileCleaner= { quartzScheduler, ctx ->
    
    	
    	try {
    		 def jobDetail = ClosureJob.createJob {
    			FileCleanerService svc = new FileCleanerService(quartzConfig)
    			svc.cleanFiles()
    		 }
                     
                     // Create a trigger for the job based on the configuration in the quartzConfig
                     // The QuartzConfig bean is available in this scope.
    		 Trigger trigger = newTrigger()
    				.withIdentity(quartzConfig?.jobName, quartzConfig?.jobGroupName)
    				.withSchedule(simpleSchedule()
    				.withIntervalInMinutes(quartzConfig?.repeatInterval)
    					.repeatForever())
    				.startAt(futureDate(quartzConfig?.startsAfter, IntervalUnit.MINUTE))
    				.build();
    		
    	
    		println("Configuring quartz job [${quartzConfig?.jobName}] for node: [${quartzConfig?.whichEnv}]")
    		quartzScheduler.scheduleJob(jobDetail, trigger)
    	}
    	catch(Exception e) {
    		e.printStackTrace(System.err)
    	}
    
    }
    

    The above specifies

  • the job that would fired ( the job created by the Closure jobDetail )
  • The trigger to fire the job (the Trigger object)
  • Attaching the job & its trigger to the scheduler
  • Create the service FileCleanerService in the services section of your grails-app (or use the grails command line to create it)

    class FileCleanerService {
    
    	static final Logger log = Logger.getLogger(this)
    	QuartzConfig cfg
    
            public FileCleanerService(QuartzConfig _cfg) {
                this.cfg = _cfg;
            }
    
    	def cleanFiles() {
    
    		Integer numThreads= cfg?.numThreads?: 4  //default to 4 if undeclared.
    		List nodes= cfg?.nodes?: [] // default to empty list if undeclared
    
    		try {
    			GParsPool.withPool(numThreads) {
    				nodes.eachParallel { node ->
    					def remote = new NodeAccessorService(node)
    					remote.wipeTheTempFiles()
    				}
    			}
    		}
    		catch(Exception e) {
    			e.printStackTrace(System.err)
    		}
    
    	}
    }
    

    The NodeAccessorService contains a method wipeTheTempFiles, which connects to the specified node (probably SSH) and runs some kind of ‘rm’ command on the temp files directory.

    After this start your grails app by issuing the ‘run-app’ command. The startup logs should indicate if the job has been triggered.

    Note:

  • Always write unit tests
  • Handle errors & recovery better
  • The old style (pre Quartz2 ) of adding a ‘job’ in the grails-app/jobs directory will also work.
  • Advertisements

    Leave a Reply

    Fill in your details below or click an icon to log in:

    WordPress.com Logo

    You are commenting using your WordPress.com account. Log Out / Change )

    Twitter picture

    You are commenting using your Twitter account. Log Out / Change )

    Facebook photo

    You are commenting using your Facebook account. Log Out / Change )

    Google+ photo

    You are commenting using your Google+ account. Log Out / Change )

    Connecting to %s

    %d bloggers like this: