Gradle Projects In Jdee

Gradle is really great automatization system. The combination of scripting language and dependency management produces simple and powerfull tool. Influenced by idea of Maven2 Jdee plugin I have made small task that generates Jdee project file. Although the script is not comprehensive (because it does not handle all Gradle features such as source sets, tasks, and so on) I think that is can be very useful, particularly in learning Gradle.

jdee.gradle

	def prj = { project ->
	"(jde-project-file-version" (["1.0"])
	"(jde-set-variables" {
	"'(jde-compile-option-directory" ([project.sourceSets.main.output.classesDir])
	"'(jde-junit-working-directory" ([project.projectDir])
	"'(jde-compile-option-source" {
	"'(" (["default"])
	}
	"'(jde-compile-option-target" {
	"'(" (["default"])
	}
	"'(jde-compile-option-command-line-args" {
	"'(" (["-${project.sourceCompatibility}"])
	}
	"'(jde-sourcepath" {
	"'(" (
	project.sourceSets.main.allSource.srcDirs
	+ project.sourceSets.test.allSource.srcDirs)
	}
	"'(jde-global-classpath" {
	"'(" (
	[] + project.sourceSets.main.output.classesDir
	+ project.sourceSets.test.output.classesDir
	+ project.sourceSets.main.allSource.srcDirs
	+ project.sourceSets.test.allSource.srcDirs
	+ (([] as Set) + project.configurations.compile.getFiles()
	+ project.configurations.testCompile.getFiles()))
	}
	}
	}
	subprojects {
	task("jdee") << {
	def output = new File(project.projectDir, "prj.el").newPrintWriter()
	try {
	prj.delegate = new NodeBuilder() {
	def lev = 0
	def write = { Object file ->
	output.print '\n' + ''.padRight(lev, ' ') + "\"${file}\"".tr('\\', '/')
	}
	Object createNode(Object name) {
	output.print '\n' + ''.padRight(lev++, ' ') + name
	return name
	}
	Object createNode(Object name, Object value) {
	createNode(name)
	value.each write
	return name
	}
	void nodeCompleted(Object parent, Object child) {
	output.print ")"
	lev--
	}
	}
	prj(project)
	output.close()
	} finally {
	output.flush()
	}
	}
	}

view raw jdee.gradle hosted with ❤ by GitHub

Save the script and apply it into your main project file (build.gradle):

apply from:'jdee.gradle'

then run the task

gradle jdee

Please note that the task is applied only for subprojects of multi-project builds. To make it works for standalone projects just substitute subprojects to projects in line 38.

Solr Functions in Action

The DataImportHandler is great contrib which provides methods to import data into Solr from relational databases. It operates in two modes "full build" and "incremental updates". Delta import calculates changed items then executes query to extract data from the source. It spawns multiple round-trips between Solr and datasource which is often an undesirable behavior. Moreover sometimes it causes "out of memory" exception. So that authors suggest an alternative way by using the same query for both full and delta updates distinguishing them with request and "dataimporter.*" parameters.

	<entity name="item" pk="ID"
	query="SELECT *
	FROM v_index
	WHERE '${dataimporter.request.clean}' != 'false'
	OR id IN
	(SELECT id
	FROM t_item i
	WHERE ('${dataimporter.request.id}' IS NULL
	AND i.updated >= TO_DATE ('${dataimporter.last_index_time}', 'yyyy-mm-dd hh24:mi:ss')))">

view raw solr.example.data-config.xml hosted with ❤ by GitHub

What's wrong here? Certainly an XML attribute isn't the best place for writing SQL text. It's better to keep SQL in files (no need to to escape characters, highlighting and so on). Second painful thing is pretty complex query. Such constructions often cause bad execution plans, especially when query runs on complicated views. So there are two options: tune query (it may become overtuned or non-portable soon) or make query simpler.

I've written a few functions. They are very simple in itself but together they produce really great cumulative effect:

• decode is remake of Oracle's decode
• load reads query from file
• run executes statements

Here is rewritten configuration file:

	<dataConfig>
	<dataSource driver="org.hsqldb.jdbcDriver" url="jdbc:hsqldb:/temp/example/ex" user="sa" />
	<function name="decode" class="solr.f.Decode"/>
	<function name="load" class="solr.f.LoadQuery"/>
	<function name="run" class="solr.f.RunQuery"/>
	<document>
	<entity
	name="item"
	query="${dataimporter.functions.load(
	dataimporter.functions.decode(
	dataimporter.request.clean,
	'false', 'delta.sql',
	'full.sql'))}">
	<field column="ID" name="id" />
	<field column="NAME" name="name" />
	<field column="MANU" name="manu" />
	<field column="WEIGHT" name="weight" />
	<field column="PRICE" name="price" />
	<field column="POPULARITY" name="popularity" />
	<field column="INSTOCK" name="inStock" />
	<field column="INCLUDES" name="includes" />
	</entity>
	</document>
	</dataConfig>

view raw data-config.xml hosted with ❤ by GitHub

This is query for full index. Note what 'before-full.sql' runs before query execution:

	/* Run material views updating
	${dataimporter.functions.run('before-full.sql')} */
	select * from v_index_full

view raw full.sql hosted with ❤ by GitHub

	begin
	refresh_mviews;
	end;

view raw before-full.sql hosted with ❤ by GitHub

Completely different table can be used in delta update:

	select *
	from
	v_index_delta
	where id in
	(select id from t_item i
	where (
	'${dataimporter.request.id}' is null
	and i.updated >= to_date(
	'${dataimporter.last_index_time}', 'yyyy-mm-dd hh24:mi:ss')))

view raw delta.sql hosted with ❤ by GitHub

Please find functions sources.

Java Development With Emacs

I would like to share my J2EE development environment. Currently it's Emacs 23 on Linux, Maven and few additional pieces of software.

JDE

The JDE is a major mode for editing java sources. I will not describe the whole package, just emphasize customizations and a way I use it. Since JDE has lack of Maven support I use Maven2 Jdee plugin to transforms pom.xml to prj.el. Here is oneliner which periodically performs such transformation on projects tree:

	$ crontab -l
	...
	0 2 * * * find ~/projects/ -path \*/arch -prune -o \( -name pom.xml -print \) | \
	sed 's/pom.xml//' | sort | awk 'BEGIN {P="doh"} {if (index($0, P)==0){P=$0; print P "pom.xml"}}' | \
	xargs -P1 -n1 $M2_HOME/bin/mvn install jdee:jdee -DskipTests=true -q -f
	...

view raw make-prj.sh hosted with ❤ by GitHub

Unfortunately this script is quite complex because there are multi-module Maven POMs and we have to exclude their subprojects.

The JDE's settings are quite simple. I just set compiler output directory to the temporary folder to prevent the source directories from useless compiled files.

	(load "jde-autoload")
	(setq jde-check-version-flag nil)
	(setq jde-compile-option-directory
	(concat user-emacs-directory "tmp"))
	(require 'gud)
	(defun java-my-minor ()
	(progn
	(gtags-mode t)
	(glasses-mode t)
	(auto-complete-mode t)
	(add-to-list 'ac-sources ac-source-gtags)
	(local-set-key [f8] 'gud-next)
	(local-set-key [f9] 'gud-cont)
	(local-set-key (kbd "M-/") 'hippie-expand)
	(local-set-key (kbd "C-c C-v .") 'jde-complete-minibuf)
	(add-hook 'before-save-hook
	(lambda ()
	(jde-import-kill-extra-imports)
	(jde-import-all)
	(jde-import-organize))
	nil t)
	(add-hook 'after-save-hook 'jde-compile nil t)))
	(add-hook 'jde-mode-hook 'java-my-minor)

view raw jde-settings.el hosted with ❤ by GitHub

Documentation

The JDE has nice feature "Context-Sensitive Class Help". There are no problems when you have graphical UI (XServer, Windows, etc.), you just specify your favorite browser (I'm sure it's Conkeror) and use it. I wish I had graphical UI. Console is all that I have. So I use text mode browser w3. It's unable to display local files so there is small fix:

	(require 'jde-help)
	;; w3 fail to load local file, so skip this feature
	(defmethod jde-jdhelper-show-url ((this jde-jdhelper) url)
	(let ((doc-url (jde-url-name url)))
	(message "Displaying %s from %s"
	(oref url :class)
	(oref (oref url :docset) :description))
	(jde-jdhelper-show-document this doc-url)))

view raw jde-help.el hosted with ❤ by GitHub

Building

One of the most boring routine in Java language is specifying imports. The JDE may do it for us automatically. All we need is just to append calls to "before-save-hook". To provide fast call back each file is compiled after saving, so you can efficiently fix cause of error without full project rebuilding.

I use "compile" command to run Maven goals. It's suitable for all targets such as building, testing, running. Torstein K. Johansen suggests to use "C-z" binding to call "compile". Setting "jde-compile" variable to "compile" seems reasonable as well. Also it's necessary to set additional regexps to parse compiler output.

I list frequently used compilations commands in special file java-compile.org which then load as compilation history. Header of each line is part of effective command so is possible to find a line in mini-buffer's history by pressing "M-r" + "HEADER":

	;; saving hooks
	(add-hook 'before-save-hook
	(lambda ()
	(jde-import-kill-extra-imports)
	(jde-import-all)
	(jde-import-organize))
	nil t)
	(add-hook 'after-save-hook 'jde-compile nil t)))
	;; recognize output
	(require 'compile)
	(setq compilation-error-regexp-alist
	(list
	;; works for maven 3.x
	'("^\\(\\[ERROR\\] \\)?\\(/[^:]+\\):\\[\\([0-9]+\\),\\([0-9]+\\)\\]" 2 3 4)
	;; works for maven jde javac server
	'("^\\(/[^:]+\\):\\([0-9]+\\):" 1 2)
	;; surefire
	'("^\\sw+(\\(\\sw+\\.\\)+\\(\\sw+\\)).+<<< \\(FAILURE\\|ERROR\\)!$"2)
	))
	;; append compilation snippets
	(let ((snippets-buf
	(find-file-noselect
	(concat user-emacs-directory "/etc/java-compile.org")))
	(setq compile-history nil)
	(with-current-buffer
	snippets-buf
	(org-map-region
	'(lambda nil
	(add-to-list
	'compile-history
	(format
	"#%s\\\n%s"
	(nth 4 (org-heading-components))
	(mapconcat 'string (org-get-entry) ""))))
	1 (buffer-end 1))
	(kill-buffer snippets-buf)))

view raw building.el hosted with ❤ by GitHub

Here is an example of predefined command lines java-compile.org:

	* ALL
	P="dev,risk"; SKIP="true"; \
	mvn clean install -f ~/projects/h/vaadin-components/pom.xml -DskipTests=$SKIP && \
	mvn clean install -f ~/projects/h/hierarchies/pom.xml -DskipTests=$SKIP -P$P
	* RUN
	P="dev,risk"; SKIP="true"; \
	MAVEN_OPTS="$MAVEN_OPTS -Xrunjdwp:transport=dt_socket,address=40487,server=y,suspend=n"; \
	mvn -pl application/ jetty:run-exploded -f ~/projects/h/hierarchies/pom.xml \
	-P$P -Djetty.port=40480
	* CRUN
	PL="core"; P="dev,risk"; SKIP="true"; \
	mvn clean install -f ~/projects/h/hierarchies/pom.xml -DskipTests=$SKIP -P$P -pl "${PL}" && \
	MAVEN_OPTS="$MAVEN_OPTS -Xrunjdwp:transport=dt_socket,address=40487,server=y,suspend=n"; \
	mvn -pl application/ jetty:run-exploded -f ~/projects/h/hierarchies/pom.xml \
	-P$P -Djetty.port=40480

view raw building.org.txt hosted with ❤ by GitHub

Tags

I use GNU GLOBAL as tagging system. It provides more correct tags table for Java then ctags or etags.

	(require 'gtags)
	(autoload 'gtags-mode "gtags" "" t)
	(setq gtags-suggested-key-mapping t)
	(global-set-key (kbd "C-c C-f") 'gtags-find-file)
	(add-hook 'gtags-select-mode-hook
	'(lambda ()
	(setq hl-line-face 'underline)
	(hl-line-mode 1)))

view raw gtags.el hosted with ❤ by GitHub

Logs

Log4j mode nice and only mode for viewing log4j logs in Emacs.

	(require 'log4j-mode)
	(setq auto-mode-alist
	(append '(("server.log" . log4j-mode)
	("catalina.out" . log4j-mode)
	("tomcat.log" . log4j-mode))
	auto-mode-alist))
	(add-hook
	'log4j-mode-hook
	(lambda ()
	(setq truncate-lines t)
	(text-scale-set -1)
	(toggle-read-only t)
	(buffer-disable-undo)
	(end-of-buffer)))

view raw log4j.el hosted with ❤ by GitHub

Completing

There are no special settings for Auto Complete Mode except adding "jde-mode" to "ac-list" and adding "gtags" as source.

	(require 'auto-complete-config)
	(ac-config-default)
	(add-to-list 'ac-dictionary-directories "~/.emacs.d/site-lisp/auto-complete/ac-dict")
	(add-to-list 'ac-modes 'jde-mode)
	(setq ac-ignore-case 'smart)
	(setq ac-use-menu-map t)
	(define-key ac-menu-map "\C-n" 'ac-next)
	(define-key ac-menu-map "\C-p" 'ac-previous)

view raw ac.el hosted with ❤ by GitHub

Debugging

I use jdb as debugger. Its default behavior is good and all what we need is to set source paths. I've made module jdb-sourcepath.el which appends source paths to rc-file (~/.jdbrc). Function jdb-sourcepath-from-rc helps us to read settings in jdb-mode:

	(require 'jdb-sourcepath)
	(add-hook
	'jdb-mode-hook
	(lambda ()
	(set 'gud-jdb-sourcepath (jdb-sourcepath-from-rc))))
	(run-at-time "1:00am" (* 60 60 24) 'jdb-setup)

view raw init-jdb-sourcepath.el hosted with ❤ by GitHub

The brilliant Jdb feature is redefining class on fly. Here is a shortcut for it:

	;; load source paths
	(add-hook
	'jdb-mode-hook
	(lambda ()
	(load-file
	(concat
	user-emacs-directory
	(convert-standard-filename "var/jdb-directories.el")))))
	;; redefine class thru jdb
	(gud-def
	gud-redefine
	(gud-call
	(format
	"redefine %%c %s/%s.class"
	(file-truename jde-compile-option-directory)
	(replace-regexp-in-string "\\." "/" (gud-format-command "%c" arg))))
	"\C-r" "Redefine class")

view raw jdb.el hosted with ❤ by GitHub

Make Jdb Sourcepath

I've reimplemented script which scans sources and makes rc file for the Jdb. Now it's in Elisp jdb-sourcepath.el.

Here is init.el snippet:

	(require 'jdb-sourcepath)
	(add-hook
	'jdb-mode-hook
	(lambda ()
	(set 'gud-jdb-sourcepath (jdb-sourcepath-from-rc))))
	(run-at-time "1:00am" (* 60 60 24) 'jdb-setup)

view raw init-jdb-sourcepath.el hosted with ❤ by GitHub

Connection Manager for the Sqlplus Mode

As a developer you have to deal with several different databases. It's not easy to keep all of your environments in mind. Let's use Org-mode to help yourself.

Keep an environment's description in "org-mode" table:

environments

service (sid)	user	pwd
prod	owner app	god
dev	dbo	love
uat	dba	secret

view raw environments.org hosted with ❤ by GitHub

Place the point to a connection line, call sqlplus-x-connect and you will get connected sqlplus-mode buffer. If different users have the same password you can write them in one cell.

Here is the sqlplus-x-connect

	(require 'sqlplus)
	(require 'org-table)
	(defvar sqlplus-x-columns '(sqlplus-x-service sqlplus-x-user sqlplus-x-pwd))
	(defun sqlplus-x-connect ()
	"Build a connection string and make a connection. The point must be in an org-mode table.
	Columns of the table must correspond to the `sqlplus-x-columns' variable."
	(interactive)
	(org-table-force-dataline)
	(let
	((cur-row (nth (org-table-current-dline) (org-table-to-lisp)))
	(is-user-selected (= (org-table-current-column) (+ 1 (position 'sqlplus-x-user sqlplus-x-columns)))))
	(sqlplus
	(format
	"%s/%s@%s"
	(if is-user-selected
	(thing-at-point 'symbol)
	(nth (position 'sqlplus-x-user sqlplus-x-columns) cur-row))
	(nth (position 'sqlplus-x-pwd sqlplus-x-columns) cur-row)
	(nth (position 'sqlplus-x-service sqlplus-x-columns) cur-row))
	(concat (nth (position 'sqlplus-x-service sqlplus-x-columns) cur-row) ".sqp"))))
	(global-set-key [f4] 'sqlplus-x-connect)

view raw sqlplus-x-connect.el hosted with ❤ by GitHub

Advice. Use EasyPG package to keep your connections table in secret.