Using Maven 2, Part 2

****************************************************************
Maven 2 Notes, Part 2: Using Remote Repositories
****************************************************************

----------------------------------------------------------------
Section 1: Putting and getting with remote repos.
----------------------------------------------------------------

* The Maven documentation covers this in detail, so I will instead look at it from a particular point of
view: how to put/get WAR files in a repository. My higher purpose is to use Maven to upload/download
war files for portlets and other web applications so that they can be deployed into Tomcat containers.

* So first, let's put our earlier war project (see part 1) into the local repository. This is pretty
simple:

mvn install

* But where is the repo? $HOME/.m2. Taking a look, you should see the directory struture


~/.m2/repository/xportlets/jobsubmit/jobsubmit-portlet/1.0-SNAPSHOT/

This directory structure is all determined by our POM header:

<project>
<modelVersion>4.0.0</modelVersion>
<groupId>xportlets.jobsubmit</groupId>
<artifactId>jobsubmit-portlet</artifactId>
<packaging>war</packaging>
<version>1.0-SNAPSHOT</version>
...
</project>

* Note that <version/> now maps to a separate directory. The war file itself is


jobsubmit-portlet-1.0-SNAPSHOT.war

So each project is required to have a specific version so that the directory strucutre can be
created properly. If you don't believe me, try deleting <version>..</version> from the above.

* Also in the repo directory that contains the above war are two additional files:

maven-metadata-local.xml
jobsubmit-portlet-1.0-SNAPSHOT.pom

I'll assume that remote repositories have a similar structure. And they do: surf around in
http://www.ibiblio.org/maven2 and you will see the equivalents, plus some additional hash files
to verify.

* The POM file in the repository, by the way, is similar to the pom.xml in your project directory. You
may notice a few differences between the original POM and the repository POM.

o All the namespaces in <project> got removed.

o Is there a POM schema? Probably not, but the .pom in your repo will be normalized a bit,
so if some of your tags are out of the canonical order, they will get moved around.

o Interestingly, <echo message="hollow world"/> got changed to <echo message="hollow world"></echo>.
OK, it isn't really that interesting.

o I assume the above is needed for some lightweight canonicalization, since these XML files must get
hashed when you put them in a remote repo.

* More importantly, the inclusion of the POM in the repository guarantees that you know all dependent jars
for a given project. This MAY mean that maven will also search for these additional jars and download them
as well. I'll have to check that out in Part X.


* You can also create an "internal" repository: this publishes your project's packaging (war, jar, etc) onto a
convient place using ssh/scp, etc. This is not to be confused with your "local" repository in ~/.m2. "Internal"
means "your group's internal repo."

Specify the internal repo in pom.xml as follows:

<project>
....
<distributionManagement>
<repository>
<id>Test</id>
<url>file:///tmp/Test/</url>
</repository>
</distributionManagement>
</project>

Note this does not mean that this repo will be used to find dependencies (?). That is done with a
<repositories/> tag, as shown below.

This specifies that we publish this to the internal repository that happens to be the local file system. Use the
command

mvn deploy

to put the results of "mvn package" into your internal repository.



This internal repository differs a bit from the normal "local" repository ~/.m2: everything is message
digested with MD5 and SHA1 digests.

ls -ltr /tmp/Test/xportlets/jobsubmit/jobsubmit-portlet/1.0-SNAPSHOT/
total 36
-rw-rw-r-- 1 gateway gateway 40 Dec 1 20:53 maven-metadata.xml.sha1
-rw-rw-r-- 1 gateway gateway 32 Dec 1 20:53 maven-metadata.xml.md5
-rw-rw-r-- 1 gateway gateway 330 Dec 1 20:53 maven-metadata.xml
-rw-rw-r-- 1 gateway gateway 40 Dec 1 20:53 jobsubmit-portlet-1.0-20051202.015334-1.war.sha1
-rw-rw-r-- 1 gateway gateway 32 Dec 1 20:53 jobsubmit-portlet-1.0-20051202.015334-1.war.md5
-rw-rw-r-- 1 gateway gateway 3564 Dec 1 20:53 jobsubmit-portlet-1.0-20051202.015334-1.war
-rw-rw-r-- 1 gateway gateway 40 Dec 1 20:53 jobsubmit-portlet-1.0-20051202.015334-1.pom.sha1
-rw-rw-r-- 1 gateway gateway 32 Dec 1 20:53 jobsubmit-portlet-1.0-20051202.015334-1.pom.md5
-rw-rw-r-- 1 gateway gateway 1203 Dec 1 20:53 jobsubmit-portlet-1.0-20051202.015334-1.pom


* You should note that you can stick any final product (ie the result of mvn deploy for a web app is
a WAR file, but your project build dependencies require jars.


----------------------------------------------------------------
Section 2: The war comes home
----------------------------------------------------------------
* Previously we saw that the war file could be placed in a "remote" repository, just like a jar. But you can't/don't
compile against War files, so we show here how to grab a War from a remote repository and put it in your local repo.


* So let's start with a new project (use an artifact to create it). We now need to modify the pom.xml in two ways:
o Specify the repository to use.
o Specify the WAR file dependency (requires non-default settings).

* Here is how you configure an "internal" repo:
<project>
...
<repositories>
<repository>
<id>blah</id>
<url>file:///tmp/Test/</url>
</repository>
</repositories>
...
</project>


Maven will look here (and Ibiblio) for your jars. One interesting question is the order that it does this. The
artifacts should be adequately named to be distinct, but download times can vary a great deal. Also, internal
repos tend to come and go, so you want to avoid the Maven 1 problem of waiting for a connection time out. This is
frustrating, especially when you do multiproject builds (and have to wait for N remote repo connection time outs).

* Anyway, the second thing you need to do is set the dependency correctly, since they default to jars. Use this:

<project>
...
<dependencies>
<dependency>
<groupId>xportlets.jobsubmit</groupId>
<artifactId>jobsubmit-portlet</artifactId>
<version>1.0-SNAPSHOT</version>
<type>war</type>
<scope>package</scope>
</dependency>
...
</dependencies>
...
</project>

That is, you must specify <type>war</type>. Note actually the scope value isn't too important here. This will
download the indicated war from the indicated repository (above) and put it in the local ~/.m2 repository. Note
that it does NOT put the WAR anywhere in the project directory (that I can tell).


* So now we want to take the war file out of the local repository and do something with it. This seems to be harder
than you might think, since Maven only pulls JARS out of the repo into the local build file, as far as I can tell.

* The solution options seem to be
o Find some mvn trick that does this;
o Write some Ant to do it; or
o Write a plugin that does it.


The third option is the way to go, probably, since the plugin increases code reusability and fills an important
general need of the maven community. It is so important and crucial that we will skip it for now.

This leaves the first and second options for now: look for a trick and let Ant do the dirty work.
If you know me, you should have had no doubt that this is what I would do. First, however, we have to take
a detour through Maven 2 properties.

----------------------------------------------------------------
Section 3: Maven 2 properties, or "What happened to ${maven.local repo}?"
----------------------------------------------------------------

* So let's put a couple of things together: we will download the war file from the "remote" repository and then
use Ant to move it to an appropriate "Tomcat" directory. We do all this in the pom.xml.


* Our first problem is accessing various built-in properties in Maven 2. The first thing I noticed (and it took me
an inordinately long amount of time to find this), is that the good old property

${maven.repo.local} //Where'd this go? I'll miss it.

is NOT set within your pom.xml. However (possibly for backward compatibility),

mvn install -Dmaven.repo.local=...

still works. Don't use this, by the way, unless you want to set up a new local repo.

* Luckily, I eventually stumbled across the answer: settings.xml. This file is located in $MAVEN_HOME/conf/. You can
edit it to localize your Maven 2 configuration, but the real value is that it lets you programmatically work with
all sorts of maven "system" properties. See

http://maven.apache.org/maven-settings/settings.html

for the big list.

* So if you want to get the directory path of your local repository, you just need to use ${settings.localRepository}.
The following pom.xml fragment (which uses the Ant plugin) can be used to verify that it works.

<project>
...
<build>
<finalName>test-webapp</finalName>
<plugins>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<phase>process-resources</phase>
<configuration>
<tasks>
<echo message="${settings.localRepository}"/>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>

Thus you now have a handle on the property that points to the repo. As we go forward a bit, we will build up
(section by section) the necessary steps to accessing a particular war.


----------------------------------------------------------------
Section 4: Setting/getting arbitrary name-value pairs with Maven 2.
----------------------------------------------------------------
* OK, now let's say you want to set an arbitrary name-value pair for use somewhere else in your POM. You
do this using build profiles (in general). See

http://maven.apache.org/guides/introduction/introduction-to-profiles.html

* For the simplest case, you can add a <profile> tag to your POM like so

<project>
...
<profiles>
<profile>
<activation>
<property>
<name>env</name>
<value>dev</value>
</property>
</activation>

<properties>
<junk.stuff>test</junk.stuff>
</properties>
</profile>
</profiles
...
</project>

I'll leave the description of this to the Maven docs, but basically the first part tells Maven when to activate
the property (the necessary condition), and the second part specifies the actual property.

So if we added

<echo message=<"${junk.stuff}"/>

to our Ant echos (see previous section), we would get the following output

[shell-prompt> mvn process-resources
...
[echo] ${junk.stuff}
...

I chose the process-resources goal (uh, phase) since it is mostly harmless.

* To actually get it to run correctly, you need to do this:

[shell-prompt> mvn process-resources -Denv=dev

This will do the thing: [echo] test

----------------------------------------------------------------
Section 5: Derived POMs and inheritence
----------------------------------------------------------------
* One of the powers of Maven 2 is that local POMs not only can derive from other local POMs, but can
also go out to remote repositories and find the parent POM.

* For example, look at section 2 in which we install the sample WAR file into an "internal" repository. Note
again "internal" repos just mean "internal to your project" to "on your local file system." Let's say we
run
[shell-prompt> mvn deploy

on our project using the /tmp/Test repository. This will place the WAR file in /tmp/Test under the appropriate
target directory. For completeness, the POM is something like this:

<project>
<modelVersion>4.0.0</modelVersion>
<groupId>xportlets.jobsubmit</groupId>
<artifactId>jobsubmit-portlet</artifactId>
<packaging>war</packaging>
<version>1.0-SNAPSHOT</version>
<name>Maven Webapp Archetype</name>
<url>http://maven.apache.org</url>
<build>
<finalName>jobsubmit-portlet</finalName>
<plugins>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<phase>process-resources</phase>
<configuration>
<tasks>
<echo message="hollow world"/>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

<distributionManagement>
<repository>
<id>Test</id>
<url>file:///tmp/Test/</url>
</repository>
</distributionManagement>

</project>


* Now let's configure a child POM that derives from this parent. We need to configure this to use our
alternative /tmp/Test repository, and then we use the <parent></parent> tags to configure things.

* The entire POM might look like this:
<project>

<!-- Define the parent -->
<parent>
<artifactId>jobsubmit-portlet</artifactId>
<groupId>xportlets.jobsubmit</groupId>
<version>1.0-SNAPSHOT</version>
</parent>

<!-- Must define a new artifactId or mvn complains -->
<artifactId>testjunk</artifactId>
<modelVersion>4.0.0</modelVersion>

<!-- This part specifies the "internal" repository -->
<repositories>
<repository>
<id>blah</id>
<url>file:///tmp/Test/</url>
</repository>
<repository>
<id>codehause</id>
<url>http://mojo.codehaus.org/maven2</url>
</repository>
</repositories>

<dependencies>
<!-- This dependency is used to get the war file -->
<dependency>
<groupId>xportlets.jobsubmit</groupId>
<artifactId>jobsubmit-portlet</artifactId>
<version>1.0-SNAPSHOT</version>
<type>war</type>
<scope>runtime</scope>
</dependency>
</dependencies>

<!-- This is the part that runs the Ant command -->
<build>
<finalName>test-webapp</finalName>
<plugins>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<phase>process-resources</phase>
<configuration>
<tasks>
<echo message="${settings.localRepository}"/>
<echo message="${pom.artifactId}"/>
<echo message="${pom.groupId}"/>
<echo message="${pom.version}"/>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>


* Basically, this means that our testjunk POM is a child of the jobsubmit-portlet. Now the cool thing is that Maven
will go to your repository and download the specified POM from the remote repository. Thus you can basically
distribute a POM like the above (ie send it to me in an email). I then run the command

[shell-prompt> mvn process-resources

This will download the job-submit SNAPSHOT war file to my local repository (because I specified it as a
dependency). This will allow me to specify the real path to the local repository (settings.localRepository),
the artifactId, versionId, and groupId.

* That is, we now at last have enough information to constuct the full path to a specific war file in the
local repository (after we download it).

* But wait,there is a problem: the output of "mvn process-resources" is

...
[echo] /home/gateway/.m2/repository
[echo] testjunk ! The problem is here.
[echo] xportlets.jobsubmit
[echo] 1.0-SNAPSHOT
....


That is, the pom.artifactId is not inherited from the POM's parent. We are stuck: each project must define its
own <artifactId>, so I can't just delete the tag, and the sensible ${parent.artifactId} property you would hope
is present is there but also set to the child's Id (ie testjunk).

* Darn.

* Well, we almost got there, but not quite. The

----------------------------------------------------------------
Section 6: Using Maven to collect and deploy remote WAR files.
----------------------------------------------------------------

* Before I get any further, let's restate the general problem: I want to use Maven to download a bunch of
wAR files (which are
really portlet apps) from various remote repositories and install them into a local Tomcat. For this particular
POM, I don't want to do
any compilation, just some simple remote WAR management.

* Our challenge now is to get out the dependency information. Unfortunately, now that there is no more Jelly,
this is a little difficult. To see, use

<echo message="${pom.dependencies}"/>

You should see something like this in your standard output when you run a mvn goal:

[echo] [Dependency {groupId=junit, artifactId=junit, version=3.8.1, type=jar},
Dependency {groupId=xportlets.jobsubmit, artifactId=jobsubmit-portlet, version=1.0-SNAPSHOT, type=war}]

In the good old Jelly days (which I refused to use), you could through this into some nested loops and
extract the information you needed (ie you could construct a real path to a jar file in the repo.)

* So what do you do in Maven 2? Again, the real answer is probably "write a plugin" but for the
current problem, I thought of a trick that I have to try. It involves multiproject builds,
so this will segue nicely into another blog.


* THE SOLUTION: we have seen how to do most of this stuff with various Maven tricks:

* Use a "shell" POM that is a child of your real POM. This lets you construct the full path to the
war file in your local repo.
* Use dependencies to download the WAR file to your local repo.
* Use a build profile to specify the correct artifact ID name.
* Use Ant to actually move the WAR file from the repo to the Tomcat webapp.


* That is, we can now embed Ant calls like following in our "shell" POM.

<copy file="${settings.locaRespository}/${groupId}/${my.parent.artifactId}/${version}/*.war"
todir="${tomcat.home}"/>

(where my.parent.artifactId is set using build profiles)

* If you are still paying attention, you might notice that I slipped on the ${groupId} as well, since
xportlets.jobsubmit maps to xportlets/jobsubmit in the repo.

* OK, the above exercise was pretty questionable, and next time I will actually write a plugin to do this,
but the above detours gave me a pretty good idead of
what Maven 2 can and cannot do.