project setup for Grails with customized plugins using git submodules

Starting with it’s initial version a couple of years ago, Grails comes with a very nice predefined project structure, e.g. domain classes go into grails-app/domain. Every artefact has its well defined location. Almost every Grails application doing a little more than a simple “Hello world” will use at some extend one or more of the 500+ available plugins. The easiest way is to add plugins by using

The plugin gets downloaded from the central repository and added into your application, either in application.properties or into grails-app/conf/BuildConfig.groovy (default for Grails 2.0). In both cases the application contains only a reference to the installed version number.

This approach works very nice as long as you don’t have to change anything inside the plugin. I’ve had multiple times the necessity to modify external plugins, due to fixing bugs or due to some special requirements that are not yet covered by the existing released version. To deal with this, there are a couple of approaches:

1) most naive:

The most simple thing to do is just opening the plugin’s source files using your IDE and make your modifications. This has some real downsides:

  • your changes will be lost whenever you install a new upstream version of the plugin
  • since Grails unpacks installed plugins by default into $HOME/.grails/<grailsversion>/projects/bullseye2/plugins/<pluginname>, your changes are not part of any SCM and only reside locally.

I’ve warned you! Don’t do this!

2) build customized plugins in a seperated location

  • download the sources of the desired plugin, unpack it outside your main application
  • modify the plugins
  • create ‘your’ version: grails package-plugin
  • switch to your application and install the generated plugin: grails install-plugin <zipfile>

This approach annoys me, since you need to repeat the ‘package-plugin’/’install-plugin’ cycle for each an every change in the plugin. So inline-plugins to the rescue…

3) use inline plugins and copy the plugin’s sources into your application’s repo

Grails has the ability to use ‘inline plugin’. There are a couple of nice blog posts covering this, so I’m just using Burt Beckwith’s slides as a reference. You basically unzip the plugin’s zip artefact into some folder inside your application (e.g. plugins folder), reference it in grails-app/conf/BuildConfig.groovy and add the extracted files to your application’s repo.
In that case you’ve basically created a diverging copy of the plugin. Whenever there’s a new release of the plugin, integrating this back into your app might become a pita. You need to reapply your changes on top of the version of the plugin. That’s doable, but requires some advanced git technique, is error prone and can consume a lot of time – I know what I’m talking about here, trust me! Another major downside is that your local fixes and improvements are not easy to contribute back to upstream.

4) use inline plugins with git submodule

At that point, I’m expecting that you’re already using git for your project. If not, NOW is the time to do this and familiarize yourself with git. It’s worth it, promised!

  • create a distinct folder inside your project, e.g. ‘plugins’ that will contain all customized plugins
  • Find out the location of the plugin’s scm. A lot of plugins host their sources now on github. If so, fork this repository on github. If it’s using svn, you could mirror it to github, see a nice blog post for this. Basically you have now a forked plugin available on github.
  • clone the forded repo into your project, I’m using the spring-security-ui plugin here as example:

    The nice thing is that your application’s repo contains now a reference to the plugin’s repo and it even remembers the sha-id of the plugin you’re currently using.
  • now add plugins/grails-spring-security-ui as a inline plugin by adding to grails-app/conf/BuildConfig.groovy
  • push your changes
  • last but not least ask your collaborators on the project to pull your changes, let them do git submodule update --init. This command is only required once for each working copy.

The really nice thing about this setup is that you can easily share your plugin changes to the upstream repository by a pullrequest. And the other way is also pretty easy: when the upstream author accepts your pullrequest and/or adds a some new functionality you can directly consume this by going to your plugin’s directory and use

NB: this results in a to-be-committed change in the upstream repo since the sha-id of the referenced repo has changed.

some notes
  • whenever you commit something to a submodule the parent repo will have a non-empty status since your copy now references another another sha-id.
  • choose the URL of the custom plugin repo carefully. If you’re the only developer make changes in the plugin, you might use the repo’s public address and override locally the push url to your private URL by

    Now there’s a difference in fetch and push URLs:

    Your collaborators will only get the public fetch URL for both and won’t be able to commit to your private repo (that’s why we’re calling it private, right?).

    The other scenario is when you want multiple developers commit to the plugin repo. In this case you need to grant them access to your private repo (or even set up a github organization for that) and use the private URL in the ‘git submodule add’ command.

Contributing your changes back to the plugin upstream repo is very well explained at Fork a repo.

Conclusion

With the approach explained you’re able to customize any plugin and track the plugin changes from your applications repo. Contributing your bugfixes and improvements back to the upstream author is a piece of cake now as well as benefiting from upstream changes.

I’d be greedly waiting for your thoughts and for discussion on this setup. Combining this with git-flow looks like a kind of best practice for Grails projects – at least for me.

8 thoughts on “project setup for Grails with customized plugins using git submodules

  1. Rodrigo Rosenfeld Rosas

    Great! I hope this could be simplified with greater support from Grails 2.0 to vendored plugins…

    On Ruby land, we have the Gemfile, in which we can specify dependencies both from published gems and from source git repositories in the same file with a similar syntax, passing directly the address of the git source. We should be able to do something like this in Grails 2.0 BuildConfig:

    compile git: ‘git://…’

  2. Serge Kreiker

    I am trying to use your grails-spring-security-ui plugin but when I add add plugins/grails-spring-security-ui as a inline plugin by adding to grails-app/conf/BuildConfig.groovy
    I get the following error

    | Error Plugin [spring-security-ui] is aliased as [grails.plugin.location.spring-security-ui] to the location [plugins/grails-spring-security-ui] in grails-app/conf/BuildConfig.groovy.
    You cannot upgrade a plugin that is configured via BuildConfig.groovy, remove the configuration to continue.

    Any idea on how to solve this issue.
    Thanks

  3. Olivier Gourment

    Hi Stefan,

    Very nice post!
    There’s something I don’t understand in the recommended setup using a git submodule. Why do you need to keep the plugins inside your Grails application? This justifies using git submodule, but in my view, you should use submodules when it’s the best and only way to use a plugin. Which it is not in Grails. You can point to an inline plugin that resides outside your app’s directory.

    I guess the question becomes, what were the problems with solution #3. I trust you that you ran into problems and it’s complicated. But at the same time, the #4 approach does not help when you have multiple Grails applications that use the customized plugin.

    Thanks

  4. admin

    Hi Olivier, thx for your feedback. The problem of #3 is that the Grails app’s git repo does not have a reference to the plugin’s repo. In other words, your app cannot make sure that a specific commit id of a plugin is used. Git submodules does exactly this. Your application can force usage of a specific commit id. I’ve found this very useful in order to have stable builds.

Leave a Reply

Your email address will not be published. Required fields are marked *