****************************************************************
Maven 2 Notes, Part 1
****************************************************************
----------------------------------------------------------------
Section 1: Getting Started
----------------------------------------------------------------
* Just download the binary and unpack. Put Maven 2's bin/ in your PATH.
* This following section is a quick review of the slightly longer intro here:
http://maven.apache.org/guides/getting-started/index.html
The Maven folks have done a good job with their introduction, so my guide below cherry-picks
the parts most relevant to me. I've also tried to supplement the official docs with some pithy observations.
* To create a new project, use the following example command. Both groupId and artifactId are required.
mvn archetype:create -DgroupId=xportlets.jobsubmit -DartifactId=jobsubmit-portlet
This will create the proper directory structure for your project:
1. The "archetype:create" goal will create a directory named after artifactId.
2. It will create a src directory like this: job-submit/src/main/xportlets/jobsubmit/
3. The tests will similarly go in src/test.
Maven 1 had issues with some directory structures matching source code packaging. It's not clear yet
if this is still true, but for now I will take the groupId to be the same as the package name, even
though this is not necessarily the case.
The "create" step unfortunately also populates your src/main and src/test directories with App.java and
AppTest.java, so you probably want to remove these.
* The pom.xml file is the maven2 equivalent of maven 1's project.xml. Fortunately, it looks very similar to the
old project.xml file format, except for a few changes:
1. Projects specify <packaging/>, which controls what happens when you use the "package" goal.
2. <dependencies/> can specify their scope: a jar needed only for testing has a scope <test/>.
* To compile your project, just use this:
mvn compile
This will put files in the usual target/ directory, as in Maven 1. Note this creates classes but does no packaging. To
create a packaged project, use
mvn package
This will first compile, then run any tests, then create a jar, war, etc, in target/. BUT what if you want to
the tests? We need this for OGCE builds, which use HttpUnit and so depend on deployment. Luckily the maven.test.skip
property is still around. Use this:
mvn package -Dmaven.test.skip=true
* To run tests by themselves, use "mvn test". If the classes have not yet been compiled, Maven will do this.
* Run "mvn clean" to clean up.
* One last note here: the various goals (compile, package, clean, etc) are all "lifecycle phases" in Maven 2. More
later.
----------------------------------------------------------------
Section 2: Making Web Applications
----------------------------------------------------------------
* Web application projects obviously have slightly different requirements for their directory structure, so to create
this, use the following convenience method.
mvn archetype:create -DgroupId=xportlets.jobsubmit -DartifactId=jobsubmit-portlet -DarchetypeArtifactId=maven-archetype-webapp
This will make the directories src/main/webapp/ and src/main/resources.
* The problem here is that it is not clear where the actual java code and java tests go. Presumably they can go under
src/main/java and src/tests as before. To test this simply, I created a dummy project (see first section) and copied
over the src/main/java and src/test directories.
This looks like it worked correctly: the resulting war file had all the classes in the right places.
* Next problems: I must specify that Jar A should be included in the WAR's WEB-INF/lib, while Jar B is needed for
compilation but not to be included in the WAR. Simple enough. See the disucssion on "scope" at
http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html.
I first used the "compile" scope--this means a) use the jar to compile, and b) include it in WEB-INF/lib. Beautiful. Your
POM dependency looks like this:
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<!-- Magic happens here -->
<scope>compile</scope>
</dependency>
</dependencies>
* Now if I use the jar to compile but do NOT want to include it in the WAR file, I do this:
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<!-- More magic -->
<scope>provided</scope>
</dependency>
</dependencies>
That is, the "provided" scope compiles with the jar but doens't include it. A quick look in the target/ directory confirms
this.
* But we need to write an archetype that will do all of this automatically.
----------------------------------------------------------------
Section 3: Integrating with Apache Ant
----------------------------------------------------------------
* Maven 2 no longer has that great, cheap maven.xml stuff that let you stick in all of your last-mile deployment stuff
with some ant. However, you can now embed Ant directly within your pom.xml file. The general overview is here:
http://maven.apache.org/guides/mini/guide-using-ant.html.
And the antrun plugin is here:
http://maven.apache.org/plugins/maven-antrun-plugin/
* Basically, the markup in your pom.xml looks something like the following. See the links above for full
exmaples.
<build>
<plugins>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<phase>process-resources</phase>
<configuration>
<tasks>
<!-- Insert arbitrary Ant -->
<echo message="hollow world"/>
</tasks>
</configuration>
<!-- Run the ant target -->
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
There are two important parts: the configuration phase and the execution phase. Your ant scriplet goes in
in the config phase, in between the targets. You must also run your scriptlet using the "run" goal.
* The mysterious <goal>run</goal> just means that we call the "run" target of the associated maven-antrun-plugin.
See (again) http://maven.apache.org/plugins/maven-antrun-plugin/.
* This is typical of Maven 2 plugin behavior, BTW.
Wednesday, November 30, 2005
Some JSF Portlet Notes
* Formatting notes: I use quotes to denote HTML tags in this blog.
* SRB and file system portlets (gridFTP, for example) would make good uses for JSF general listing/tree view
classes.
* The point is that both of the above have the same general abstract file view of life and so can be repeated.
* UISelectItems is potentially important, since it holds entire groups of values. The display of the values
seems to be tied to the component that surrounds it. For example, the HtmlSelectManyListbox always renders these
with "select" and "options" and "optgroups" if there are multiple values. There are several display objects
associated with these groups (select boxes, checkboxes, radios, etc.)
* HtmlDataTable is a just a "table" at its heart. But it does lots of fancy output--basically you can line up
your mutliple part data into columns and have JSF figure out the for loops.]
* But the question now is this: how do you need to extend HtmlDataTable goodness? Maybe you don't and the
whole issue is mapping this to a backing bean.
* Also, note again that the backing bean can be independent of the display. So for example a tree view of a
directory and a table view (as per current gridftp) are two different views. So a neat trick would be to
convert between one view and another.
* Another common data model may be the "user session data model" in which the user wants to navigate a
logical tree of results. This may be output as either tables with a top navigation link (as per gridftp) or
it may be the common tree view, or so on.
* So the currently unanwsered question is this: what is the data model for a big glob of results. If I do a
gridftp on a directory, how to I abstractly handle the results? That is, I have to map this into some
HTMLDataTable or Tree or something at some point in the future.
* Basic problem statement: portlets are good at some level, but they are not fine-grained enough. For example,
a job submission portlet is marginally useful since what you really want to do is make an application submission
portlet. A GridFTP portlet has some usefulness, but what you really want to do is collect all the files
(input, output, stdout, error, etc) into a single grouping and associate this with common GridFTP-like
tasks. That is, there is a logical filespace associated with a specific application. This may include other
stuff like metadata.
Similarly, SRB is a file system in its own right. So I need to be able to navigate this. But back to the
basic problem: how do I build an application page this way?
I cannot build a generic portlet that is an abstract version of a OGSA-DAI client. This doesn't have
much usefulness. Instead what I would like is a set of widgets and data models that can hold generic
OGSA-DAI stuff in them. These can then be used to build up specific OGSA-DAI application portlets.
* Another issue is this: how do I build up a workflow page? This is not really intended to be a workflow
composer but rather a lower level thing: I often need to associate the backend resources (code, data) with
each other. This is a common task.
* Yet another common job: need a list of all my jobs (past, pending, current, etc). This should be an
abstract view (an arbitrary collection) that gets mapped to a different view (tables, trees, etc.). These
view objects can be associated with events, where events (action events) are really COMMON GRID TASKS.
* So one can think of an ACTION EVENT LISTENER of type "run job" or of type "download file" or of type
"remove value", and so on. These are common actions. There can also be other action events associated with
a particular instance.
* Another possible use: the pages may be constructed using Sakai-like standard components for a common
look to everything. So we can adopt/modify this for a standard OGCE look.
* I think the problem with the current approach is that we confuse ACTIONS with PORTLETS. For example, the GRAM
portlet is really a single action portlet (for the most part): do the thing. GridFTP is a collection of actions
for remote file manipulation, and so on. In the current (velocity) set up, these actions are tied to specific
forms by name. But this is too rigid. I can't (for example) combine the action code a button to multiple
listeners.
* Some JSF virtues:
- Multiple views to same data: my GridFTP view can toggle between Tree and Table views.
- Reusable views for different data: GridFTP, SRB, Job Monitors, OGSA-DAI can all reuse my nift tree widget.
- Multiple event listeners on a single form. Compare this to Velocity, JSP, which basically have
1 action per form. Consider the problem of reusing the action code of a velocity portlet.
- Input form names are auto-generated, so you don't have to keep track of these or break your
portlet if you change names. Or, you don't have to adopt naming conventions to reuse your action
code. Consider again the Velocity example.
- Lots of built-in validators (for strings, ints, etc, or for ranges of values, and so on).
- We can create libraries of stuff: events, nifty display widgets, etc.
* Validators: can write custom validators to make sure that the user has a credential.
* SRB and file system portlets (gridFTP, for example) would make good uses for JSF general listing/tree view
classes.
* The point is that both of the above have the same general abstract file view of life and so can be repeated.
* UISelectItems is potentially important, since it holds entire groups of values. The display of the values
seems to be tied to the component that surrounds it. For example, the HtmlSelectManyListbox always renders these
with "select" and "options" and "optgroups" if there are multiple values. There are several display objects
associated with these groups (select boxes, checkboxes, radios, etc.)
* HtmlDataTable is a just a "table" at its heart. But it does lots of fancy output--basically you can line up
your mutliple part data into columns and have JSF figure out the for loops.]
* But the question now is this: how do you need to extend HtmlDataTable goodness? Maybe you don't and the
whole issue is mapping this to a backing bean.
* Also, note again that the backing bean can be independent of the display. So for example a tree view of a
directory and a table view (as per current gridftp) are two different views. So a neat trick would be to
convert between one view and another.
* Another common data model may be the "user session data model" in which the user wants to navigate a
logical tree of results. This may be output as either tables with a top navigation link (as per gridftp) or
it may be the common tree view, or so on.
* So the currently unanwsered question is this: what is the data model for a big glob of results. If I do a
gridftp on a directory, how to I abstractly handle the results? That is, I have to map this into some
HTMLDataTable or Tree or something at some point in the future.
* Basic problem statement: portlets are good at some level, but they are not fine-grained enough. For example,
a job submission portlet is marginally useful since what you really want to do is make an application submission
portlet. A GridFTP portlet has some usefulness, but what you really want to do is collect all the files
(input, output, stdout, error, etc) into a single grouping and associate this with common GridFTP-like
tasks. That is, there is a logical filespace associated with a specific application. This may include other
stuff like metadata.
Similarly, SRB is a file system in its own right. So I need to be able to navigate this. But back to the
basic problem: how do I build an application page this way?
I cannot build a generic portlet that is an abstract version of a OGSA-DAI client. This doesn't have
much usefulness. Instead what I would like is a set of widgets and data models that can hold generic
OGSA-DAI stuff in them. These can then be used to build up specific OGSA-DAI application portlets.
* Another issue is this: how do I build up a workflow page? This is not really intended to be a workflow
composer but rather a lower level thing: I often need to associate the backend resources (code, data) with
each other. This is a common task.
* Yet another common job: need a list of all my jobs (past, pending, current, etc). This should be an
abstract view (an arbitrary collection) that gets mapped to a different view (tables, trees, etc.). These
view objects can be associated with events, where events (action events) are really COMMON GRID TASKS.
* So one can think of an ACTION EVENT LISTENER of type "run job" or of type "download file" or of type
"remove value", and so on. These are common actions. There can also be other action events associated with
a particular instance.
* Another possible use: the pages may be constructed using Sakai-like standard components for a common
look to everything. So we can adopt/modify this for a standard OGCE look.
* I think the problem with the current approach is that we confuse ACTIONS with PORTLETS. For example, the GRAM
portlet is really a single action portlet (for the most part): do the thing. GridFTP is a collection of actions
for remote file manipulation, and so on. In the current (velocity) set up, these actions are tied to specific
forms by name. But this is too rigid. I can't (for example) combine the action code a button to multiple
listeners.
* Some JSF virtues:
- Multiple views to same data: my GridFTP view can toggle between Tree and Table views.
- Reusable views for different data: GridFTP, SRB, Job Monitors, OGSA-DAI can all reuse my nift tree widget.
- Multiple event listeners on a single form. Compare this to Velocity, JSP, which basically have
1 action per form. Consider the problem of reusing the action code of a velocity portlet.
- Input form names are auto-generated, so you don't have to keep track of these or break your
portlet if you change names. Or, you don't have to adopt naming conventions to reuse your action
code. Consider again the Velocity example.
- Lots of built-in validators (for strings, ints, etc, or for ranges of values, and so on).
- We can create libraries of stuff: events, nifty display widgets, etc.
* Validators: can write custom validators to make sure that the user has a credential.
Some Globus 4 Notes
* Install as normal. Note however that all of the Globus WS stuff depends up on a postgres DB being properly configured. See the instructions
under RFT manual at http://www.globus.org/toolkit/docs/4.0/admin/docbook/ch10.html. If you don't get this running, then your globus
remote command executions will work but they will not clean up properly and so will never exit.
* TO RESTART POSTGRES SERVER: this is needed to do anything with the ws-gram container. The steps are these:
1. Login to gf1 as the user "postgres".
2. Use the installation in the directory /usr/local/pgsql. There is another installation at /var/lib that should be ignored.
3. Set this environment variable: export PGDATA=/usr/local/pgsql/data.
4. Run the command "/usr/bin/pg_ctl start -o -i" as the postgres user. If everything is OK, you will get something like this:
bash-2.05b$ pg_ctl start -o -i
pg_ctl: Another postmaster may be running. Trying to start postmaster anyway.
LOG: database system was interrupted at 2005-06-08 18:13:55 EST
LOG: checkpoint record is at 0/873128
LOG: redo record is at 0/873128; undo record is at 0/0; shutdown FALSE
LOG: next transaction id: 1543; next oid: 25213
LOG: database system was not properly shut down; automatic recovery in progress
LOG: ReadRecord: record with zero length at 0/873168
LOG: redo is not required
postmaster successfully started
bash-2.05b$ LOG: database system is ready
5. Test to make sure the rftDatabase is running correctly. Use the command "psql rftDatabase". You should see
bash-2.05b$ psql rftDatabase
Welcome to psql 7.3.4-RH, the PostgreSQL interactive terminal.
Type: \copyright for distribution terms
\h for help with SQL commands
\? for help on internal slash commands
\g or terminate with semicolon to execute query
\q to quit
rftDatabase=#
6. The GF1 database is configured to work with the user "manacar" so su manacar. Set up your globus environment as necessary:
"export GLOBUS_LOCATION=/home/globus/nmi-all-7.0-red9/" and then "source $GLOBUS_LOCATION/etc/globus-user-env.sh".
7. Start the container as user manacar: "globus-start-container".
under RFT manual at http://www.globus.org/toolkit/docs/4.0/admin/docbook/ch10.html. If you don't get this running, then your globus
remote command executions will work but they will not clean up properly and so will never exit.
* TO RESTART POSTGRES SERVER: this is needed to do anything with the ws-gram container. The steps are these:
1. Login to gf1 as the user "postgres".
2. Use the installation in the directory /usr/local/pgsql. There is another installation at /var/lib that should be ignored.
3. Set this environment variable: export PGDATA=/usr/local/pgsql/data.
4. Run the command "/usr/bin/pg_ctl start -o -i" as the postgres user. If everything is OK, you will get something like this:
bash-2.05b$ pg_ctl start -o -i
pg_ctl: Another postmaster may be running. Trying to start postmaster anyway.
LOG: database system was interrupted at 2005-06-08 18:13:55 EST
LOG: checkpoint record is at 0/873128
LOG: redo record is at 0/873128; undo record is at 0/0; shutdown FALSE
LOG: next transaction id: 1543; next oid: 25213
LOG: database system was not properly shut down; automatic recovery in progress
LOG: ReadRecord: record with zero length at 0/873168
LOG: redo is not required
postmaster successfully started
bash-2.05b$ LOG: database system is ready
5. Test to make sure the rftDatabase is running correctly. Use the command "psql rftDatabase". You should see
bash-2.05b$ psql rftDatabase
Welcome to psql 7.3.4-RH, the PostgreSQL interactive terminal.
Type: \copyright for distribution terms
\h for help with SQL commands
\? for help on internal slash commands
\g or terminate with semicolon to execute query
\q to quit
rftDatabase=#
6. The GF1 database is configured to work with the user "manacar" so su manacar. Set up your globus environment as necessary:
"export GLOBUS_LOCATION=/home/globus/nmi-all-7.0-red9/" and then "source $GLOBUS_LOCATION/etc/globus-user-env.sh".
7. Start the container as user manacar: "globus-start-container".
Some Historical GridFTP Issues
* Summary:
- The CoG's file listing GridFTP operations don't work with GridFTP 2.0 servers in GT 4.
- PWD is flaky (sometimes works, sometimes not).
* GridFTP Installations
- palermo.ucs.indiana.edu runs GridFTP v 1.17 from NMI R5 (~ 1 year old).
- danube.ucs.indiana.edu runs GridFTP v 2.0 from the GT4 release (from globus.org).
- gf1.ucs.indiana.edu runs GridFTP v 2.0 (the same) from the NMI R7 release.
* Set up:
- I run all servers in the standard way, as root, started by xinet.d.
- I checked installations using the globus-url-copy tool in various combinations
(gf1 connect to danube, danube to palermo, danube to gf1). After lots of fun with
certificates and signing policies, it all works.
- Note of course this does not let me verify the list operation that causes problems.
* Here is my cog-kit set up:
- I use Mike's maven stuff to download all cog-jars.
- After unpacking, I run
maven ogceDeploy:ogceDeploy -Dtomcat.home=$HOME/GridFTPTest/test-shared-libs
- I set the classpath to point to the bazillion jars in the shared/lib that gets created.
- I then compile the demo FileOperations.java program from the CoGKit website
(I downloaded it directly from the website).
* Sanity check:
- I run several commands successfully against the old GridFTP 1.17 server on palermo.
- OK, rmdir doesn't work but I may just be using it incorrectly.
* Now for GridFTP on gf1. The following work OK.
- cd works (verified with mkdir)
- mkdir
- rmdir (actually doesn't work but doesn't report errors either).
- isDirectory
- exists
- rename
- putfile
- getfile
* Below are the commands that cause problems. These are the same for both the NMI R7 server
and the GT4 version (not unexpected since they are identical versions).
---------------------------------------------------------------------------------
* Problem: ls
---------------------------------------------------------------------------------
Please Enter your command with its arguments
ls
- Control channel sending: PASV
- Control channel received: 227 Entering Passive Mode (156,56,104,81,145,163)
- Control channel sending: LIST -d *
- Getting default credential
Operation failed: File operation failed
Submission Exception: File Operation failed
Please Enter your command with its arguments
----------------------------------------------------------------------------------
----------------------------------------------------------------------------------
Problem: ls
----------------------------------------------------------------------------------
Please Enter your command with its arguments
ls /home/gateway
- Control channel sending: PWD
- Control channel received: 257 "/home/gateway" is current directory.
- Control channel sending: CWD /home/gateway
- Control channel received: 257 "/home/gateway" is current directory.
- Control channel sending: PASV
- Control channel received: 250 CWD command successful.
Operation failed: File operation failed
Submission Exception: File Operation failed
Please Enter your command with its arguments
----------------------------------------------------------------------------------
----------------------------------------------------------------------------------
Problem: PWD. Actually, this always seems to fail when used immediately
after another client command (ls, cd, etc), but then works the second time.
----------------------------------------------------------------------------------
Please Enter your command with its arguments
pwd
- Control channel sending: PWD
- Control channel received: 227 Entering Passive Mode (156,56,104,81,145,164)
Operation failed: File operation failed
Submission Exception: File Operation failed
Please Enter your command with its arguments
----------------------------------------------------------------------------------
----------------------------------------------------------------------------------
- The CoG's file listing GridFTP operations don't work with GridFTP 2.0 servers in GT 4.
- PWD is flaky (sometimes works, sometimes not).
* GridFTP Installations
- palermo.ucs.indiana.edu runs GridFTP v 1.17 from NMI R5 (~ 1 year old).
- danube.ucs.indiana.edu runs GridFTP v 2.0 from the GT4 release (from globus.org).
- gf1.ucs.indiana.edu runs GridFTP v 2.0 (the same) from the NMI R7 release.
* Set up:
- I run all servers in the standard way, as root, started by xinet.d.
- I checked installations using the globus-url-copy tool in various combinations
(gf1 connect to danube, danube to palermo, danube to gf1). After lots of fun with
certificates and signing policies, it all works.
- Note of course this does not let me verify the list operation that causes problems.
* Here is my cog-kit set up:
- I use Mike's maven stuff to download all cog-jars.
- After unpacking, I run
maven ogceDeploy:ogceDeploy -Dtomcat.home=$HOME/GridFTPTest/test-shared-libs
- I set the classpath to point to the bazillion jars in the shared/lib that gets created.
- I then compile the demo FileOperations.java program from the CoGKit website
(I downloaded it directly from the website).
* Sanity check:
- I run several commands successfully against the old GridFTP 1.17 server on palermo.
- OK, rmdir doesn't work but I may just be using it incorrectly.
* Now for GridFTP on gf1. The following work OK.
- cd
- mkdir
- rmdir (actually doesn't work but doesn't report errors either).
- isDirectory
- exists
- rename
- putfile
- getfile
* Below are the commands that cause problems. These are the same for both the NMI R7 server
and the GT4 version (not unexpected since they are identical versions).
---------------------------------------------------------------------------------
* Problem: ls
---------------------------------------------------------------------------------
Please Enter your command with its arguments
ls
- Control channel sending: PASV
- Control channel received: 227 Entering Passive Mode (156,56,104,81,145,163)
- Control channel sending: LIST -d *
- Getting default credential
Operation failed: File operation failed
Submission Exception: File Operation failed
Please Enter your command with its arguments
----------------------------------------------------------------------------------
----------------------------------------------------------------------------------
Problem: ls
----------------------------------------------------------------------------------
Please Enter your command with its arguments
ls /home/gateway
- Control channel sending: PWD
- Control channel received: 257 "/home/gateway" is current directory.
- Control channel sending: CWD /home/gateway
- Control channel received: 257 "/home/gateway" is current directory.
- Control channel sending: PASV
- Control channel received: 250 CWD command successful.
Operation failed: File operation failed
Submission Exception: File Operation failed
Please Enter your command with its arguments
----------------------------------------------------------------------------------
----------------------------------------------------------------------------------
Problem: PWD. Actually, this always seems to fail when used immediately
after another client command (ls, cd, etc), but then works the second time.
----------------------------------------------------------------------------------
Please Enter your command with its arguments
pwd
- Control channel sending: PWD
- Control channel received: 227 Entering Passive Mode (156,56,104,81,145,164)
Operation failed: File operation failed
Submission Exception: File Operation failed
Please Enter your command with its arguments
----------------------------------------------------------------------------------
----------------------------------------------------------------------------------
SVN Notes
* Installed svn on gf2.ucs.indiana.edu with "yum install svn". First time to use yum, much better than unadorned rpm.
* Logged into pamd.csit.fsu.edu, created a new repository with
svnadmin create $HOME/VLAB_SVN
* Imported vlab source with svn import vlab file:///home/myname/VLAB_SVN/vlab.
* Also was able to list files: svn list file:///home/myname/VLAB_SVN/vlab.
* After fooling around for a bit, I got SVN to work over ssh from gf2. Since my usernames on pamd and gf2 are not the same, I had to use
export SVN_SSH="ssh -l myname"
on the command line. Then the usual SVN command worked:
svn list svn+ssh://pamd.csit.fsu.edu/home/myname/VLAB_SVN/vlab
* Checked out on gf2 successfully with
svn checkout svn+ssh://pamd.csit.fsu.edu/home/myname/VLAB_SVN/vlab
* I deleted the unnecessary "target" directory with the commands
svn delete target svn+ssh://pamd.csit.fsu.edu/home/myname/VLAB_SVN/vlab
svn commit -m "Deleting build directory"
* The "vlab" group already exists on pamd, so I changed group ownerships to that. I then had to set the group bit with
chmod -R g+s .
This means that all new files created in any subdirectories are created with "vlab" group ownership.
* Finally, the umask needs to be set correctly. I used
umask 007
since this gives group write permission. But it is a problem since my default group is "visitor" so anyone in "visitor" can
create files in my account. I think this needs to go in each user's .login or .cshrc file. I got around this by changing all of
my group permissions to "vlab".
* Looks like a cheap way to set permissions is to alias the svn command to "umask 007; svn". This must be done on each client
machine, however, whereas the "put umask 007 in the .login file" requires less work.
* Logged into pamd.csit.fsu.edu, created a new repository with
svnadmin create $HOME/VLAB_SVN
* Imported vlab source with svn import vlab file:///home/myname/VLAB_SVN/vlab.
* Also was able to list files: svn list file:///home/myname/VLAB_SVN/vlab.
* After fooling around for a bit, I got SVN to work over ssh from gf2. Since my usernames on pamd and gf2 are not the same, I had to use
export SVN_SSH="ssh -l myname"
on the command line. Then the usual SVN command worked:
svn list svn+ssh://pamd.csit.fsu.edu/home/myname/VLAB_SVN/vlab
* Checked out on gf2 successfully with
svn checkout svn+ssh://pamd.csit.fsu.edu/home/myname/VLAB_SVN/vlab
* I deleted the unnecessary "target" directory with the commands
svn delete target svn+ssh://pamd.csit.fsu.edu/home/myname/VLAB_SVN/vlab
svn commit -m "Deleting build directory"
* The "vlab" group already exists on pamd, so I changed group ownerships to that. I then had to set the group bit with
chmod -R g+s .
This means that all new files created in any subdirectories are created with "vlab" group ownership.
* Finally, the umask needs to be set correctly. I used
umask 007
since this gives group write permission. But it is a problem since my default group is "visitor" so anyone in "visitor" can
create files in my account. I think this needs to go in each user's .login or .cshrc file. I got around this by changing all of
my group permissions to "vlab".
* Looks like a cheap way to set permissions is to alias the svn command to "umask 007; svn". This must be done on each client
machine, however, whereas the "put umask 007 in the .login file" requires less work.
uPortal Notes
* Started with vanilla install of the Quick Start version. I recommend doing this since you a) realize you have
to run the DB server separately (not well documented in the full build stuff), and b) you can see what it is
supposed to do.
* Doing vanilla install of the "uportal only" distribution of 2.3.3.
* Got Hypersonic from http://sourceforge.net/projects/hsqldb.
* Had some documentation issues. Finally settled on the instructions in the distribution's
README, but also the ones below. The README worked but had an important omission about starting the
database as a separate process. You need to run this command from the hsqldb/lib directory:
java -cp hsqldb.jar org.hsqldb.Server -database uPortal2 -port 8887
* Note you should (always) explicitly state your classpath (as in the above line) or you will regret it.
* You also must change out the version of hsqldb.jar in the uportal lib directory if using hsqldb-1.7 or later.
* For some reason, I had problems with localhost: I had to change localhost in rdbm.properties's jdbcUrl variable
to the full machine name before uPortal would build.
* The other installation documents had problems--wrong version of Tomcat, etc.
- Found lots of documentation at http://www.uportal.org/administrators/index.html, using
http://www.fair-portal.hull.ac.uk/downloads/uPortalGuide1.pdf. But it had some problems--for
the pre-JSR 168, version.
- There is also the install guide from http://www.uportal.org/administrators/install_v2.html, but it
suffers from excessive hyperlinks, didn't have a printer friendly version of the whole thing.
* Also ran into some Tomcat issues: 5.0.19 DID NOT WORK but 5.0.27 was OK. I note that 5.0.18 comes with
Quick Start, so 5.0.19 must have broken something. And in fact I now see everywhere (including the Pluto site)
that it has a well known bug.
* The portlet war files are located in lib/portlets and include the pluto testsuite and some other stuff. These
do not include the Pluto testsuite source code, so to look at this I had to get them from the pluto site using
[prompt> cvs -d :pserver:anoncvs@cvs.apache.org:/home/cvspublic login
[prompt> cvs -z3 -d :pserver:anoncvs@cvs.apache.org:/home/cvspublic co jakarta-pluto
* Useful Links: there are tons of documentation, a lot of which is on non-jasisg sites, so you should download and save
locally--the links were often broken while I was surfing. Of all the stuff, I found the following useful for
these specific things:
a) Adding tabs, portlets/channels, and content: http://mis105.mis.udel.edu/ja-sig/docs/uPortal_Documentation_Chapter_1.pdf.
Quick and dumb. Don't forget to save those changes.
b) Changing the ugly default (guest) layout and adding users: http://www.uportal.org/implementors/usermgmt.html
c) 168 Portlets in uPortal: http://www.uportal.org/implementors/portlets/workingWithPortlets.html
* Adding portlets is a bit tricky, as described in the above link. You have to get the name correct when editing uPortal's
channels. This is somewhat scruffy:
a) The first set of forms, used for naming the portal, don't matter too much and are probably for human consumption.
b) The second set of forms, "portlet definition", has to be done this way: [war_file_name].[porlet_name], where
[war_file_name] is the name of the war/webapp that contains the portlet code, and portlet_name is the value
of the value of portlet.xml's
c) So for the proxy manager portlet, proxymanager-portlet.war, the name to enter into the uportal form to propertly
create this channel is "proxymanager-portlet.ProxyManagerPortlet".
* Experimenting with adding a portlet: edited the testsuite's portlet.xml file to include TestPortlet3, a clone of TestPorlet1 and 2.
- Restarted Tomcat just in case and logged in as developer/developer.
- Added a new, empty tab for this.
- Clicked the channel manager and went through the steps
- Then edited the blank tab to add the new portlet/channel (it appeared correctly on the list).
- Don't forget to save the changes.
* And it failed--skip ahead to see how to do this correctly.
I got this error:
Channel ID: 53
Message: IChannelRenderer.completeRendering() threw
Error type: Channel failed to render (code 1)
Problem type:
Error message Initialization of the portlet container failed.
* The stackTrace gave
java.lang.NullPointerException
at org.apache.pluto.invoker.impl.PortletInvokerImpl.invoke(PortletInvokerImpl.java:109)
at org.apache.pluto.invoker.impl.PortletInvokerImpl.load(PortletInvokerImpl.java:80)
at org.apache.pluto.PortletContainerImpl.portletLoad(PortletContainerImpl.java:205)
at org.jasig.portal.channels.portlet.CPortletAdapter.initPortletWindow(CPortletAdapter.java:261)
at org.jasig.portal.channels.portlet.CPortletAdapter.renderCharacters(CPortletAdapter.java:453)
at org.jasig.portal.MultithreadedCharacterChannelAdapter.renderCharacters(MultithreadedCharacterChannelAdapter.java:71)
at org.jasig.portal.ChannelRenderer$Worker.run(ChannelRenderer.java:481)
at org.jasig.portal.utils.threading.Worker.run(Worker.java:88)
* Neither logging out/in nor restarting the server helped.
* Also, I could add TestPortal2 with no problem.
* Noticed I had added two different versions of Test Portal #3. Redid everything and now get
Channel ID: 61
Message:IChannelRenderer.completeRendering() threw
Error type: Channel failed to render (code 1)
Problem type:
Error message Initialization of the portlet container failed.
* The stack trace from Pluto was essentially the same as above, despite the different uPortal error. In fact, the
same Pluto error seemed to cause several different uPortal errors.
* So, after a tip from Eric, I redeployed and that worked. I unpacked testsuite.war, edited the portlet.xml file to
add test portlet #3, recreated the war (jar -cf) and redeployed using
ant deployPortletApp -DportletApp=./lib/portlets/testsuite.war
(the usual way) from the uPortal directory. Restarted Tomcat, published the channel, then subscribed and all was well.
to run the DB server separately (not well documented in the full build stuff), and b) you can see what it is
supposed to do.
* Doing vanilla install of the "uportal only" distribution of 2.3.3.
* Got Hypersonic from http://sourceforge.net/projects/hsqldb.
* Had some documentation issues. Finally settled on the instructions in the distribution's
README, but also the ones below. The README worked but had an important omission about starting the
database as a separate process. You need to run this command from the hsqldb/lib directory:
java -cp hsqldb.jar org.hsqldb.Server -database uPortal2 -port 8887
* Note you should (always) explicitly state your classpath (as in the above line) or you will regret it.
* You also must change out the version of hsqldb.jar in the uportal lib directory if using hsqldb-1.7 or later.
* For some reason, I had problems with localhost: I had to change localhost in rdbm.properties's jdbcUrl variable
to the full machine name before uPortal would build.
* The other installation documents had problems--wrong version of Tomcat, etc.
- Found lots of documentation at http://www.uportal.org/administrators/index.html, using
http://www.fair-portal.hull.ac.uk/downloads/uPortalGuide1.pdf. But it had some problems--for
the pre-JSR 168, version.
- There is also the install guide from http://www.uportal.org/administrators/install_v2.html, but it
suffers from excessive hyperlinks, didn't have a printer friendly version of the whole thing.
* Also ran into some Tomcat issues: 5.0.19 DID NOT WORK but 5.0.27 was OK. I note that 5.0.18 comes with
Quick Start, so 5.0.19 must have broken something. And in fact I now see everywhere (including the Pluto site)
that it has a well known bug.
* The portlet war files are located in lib/portlets and include the pluto testsuite and some other stuff. These
do not include the Pluto testsuite source code, so to look at this I had to get them from the pluto site using
[prompt> cvs -d :pserver:anoncvs@cvs.apache.org:/home/cvspublic login
[prompt> cvs -z3 -d :pserver:anoncvs@cvs.apache.org:/home/cvspublic co jakarta-pluto
* Useful Links: there are tons of documentation, a lot of which is on non-jasisg sites, so you should download and save
locally--the links were often broken while I was surfing. Of all the stuff, I found the following useful for
these specific things:
a) Adding tabs, portlets/channels, and content: http://mis105.mis.udel.edu/ja-sig/docs/uPortal_Documentation_Chapter_1.pdf.
Quick and dumb. Don't forget to save those changes.
b) Changing the ugly default (guest) layout and adding users: http://www.uportal.org/implementors/usermgmt.html
c) 168 Portlets in uPortal: http://www.uportal.org/implementors/portlets/workingWithPortlets.html
* Adding portlets is a bit tricky, as described in the above link. You have to get the name correct when editing uPortal's
channels. This is somewhat scruffy:
a) The first set of forms, used for naming the portal, don't matter too much and are probably for human consumption.
b) The second set of forms, "portlet definition", has to be done this way: [war_file_name].[porlet_name], where
[war_file_name] is the name of the war/webapp that contains the portlet code, and portlet_name is the value
of the value of portlet.xml's
c) So for the proxy manager portlet, proxymanager-portlet.war, the name to enter into the uportal form to propertly
create this channel is "proxymanager-portlet.ProxyManagerPortlet".
* Experimenting with adding a portlet: edited the testsuite's portlet.xml file to include TestPortlet3, a clone of TestPorlet1 and 2.
- Restarted Tomcat just in case and logged in as developer/developer.
- Added a new, empty tab for this.
- Clicked the channel manager and went through the steps
- Then edited the blank tab to add the new portlet/channel (it appeared correctly on the list).
- Don't forget to save the changes.
* And it failed--skip ahead to see how to do this correctly.
I got this error:
Channel ID: 53
Message: IChannelRenderer.completeRendering() threw
Error type: Channel failed to render (code 1)
Problem type:
Error message Initialization of the portlet container failed.
* The stackTrace gave
java.lang.NullPointerException
at org.apache.pluto.invoker.impl.PortletInvokerImpl.invoke(PortletInvokerImpl.java:109)
at org.apache.pluto.invoker.impl.PortletInvokerImpl.load(PortletInvokerImpl.java:80)
at org.apache.pluto.PortletContainerImpl.portletLoad(PortletContainerImpl.java:205)
at org.jasig.portal.channels.portlet.CPortletAdapter.initPortletWindow(CPortletAdapter.java:261)
at org.jasig.portal.channels.portlet.CPortletAdapter.renderCharacters(CPortletAdapter.java:453)
at org.jasig.portal.MultithreadedCharacterChannelAdapter.renderCharacters(MultithreadedCharacterChannelAdapter.java:71)
at org.jasig.portal.ChannelRenderer$Worker.run(ChannelRenderer.java:481)
at org.jasig.portal.utils.threading.Worker.run(Worker.java:88)
* Neither logging out/in nor restarting the server helped.
* Also, I could add TestPortal2 with no problem.
* Noticed I had added two different versions of Test Portal #3. Redid everything and now get
Channel ID: 61
Message:IChannelRenderer.completeRendering() threw
Error type: Channel failed to render (code 1)
Problem type:
Error message Initialization of the portlet container failed.
* The stack trace from Pluto was essentially the same as above, despite the different uPortal error. In fact, the
same Pluto error seemed to cause several different uPortal errors.
* So, after a tip from Eric, I redeployed and that worked. I unpacked testsuite.war, edited the portlet.xml file to
add test portlet #3, recreated the war (jar -cf) and redeployed using
ant deployPortletApp -DportletApp=./lib/portlets/testsuite.war
(the usual way) from the uPortal directory. Restarted Tomcat, published the channel, then subscribed and all was well.
JSR 168 Portlet Notes
* These are written to be used with uPortal 2.3.3 or so, but they should be general.
* I'm taking this as the directory structure
src---build.xml [ant file for building, deploying, and warring the portlet
|
--external_libs [pluto/portlet jar files not to be included in the war]
|
--java [portlet source code]
|
--webapp [the deploy directory that gets warred up at the end]
The webapp directory has this structure:
webapp---images
|
--jsp [jsp pages loaded by the portlet]
|
--WEB-INF [classes, lib, web.xml, portlet.xml]
* If using ant, make sure you read the task documents carefully. The directory
structure it creates may not match your source directory--you have to specify lib and classes
explicily.
* I wasted a lot of time trying to deploy a simple portlet into uPortal and finally
found that my web.xml file had a problem. I originally copied this over from the
testsuite's web.xml file:
Junk Testsuite
tomcat
The uPortal tool "ant deployPortletApp" constantly gave "Exception in thread "main" java.lang.StackOverflowError".
This is in the Deployer.java file from uPortal, in the prepareWebArchive() method. The lines
WebApplicationUnmarshaller wau = new WebApplicationUnmarshaller();
wau.init(new FileInputStream(webXml), webModule);
were the source of the error--probably this wraps some castor-like thing that processes the xml. I don't know why it
failed for my test portlet but worked for the testsuite. Anyway, I changed this to
Junk Testsuite
and it worked like a charm.
* I'm doing everything below as uPortal's "admin" user. I don't know or want to learn the uPortal channel control
permissions yet.
* After deploying the portlet, I can add it using the channel manager. Had problems because I did not read the
(uPortal portlet) instructions carefully as usual. You must make sure that you give the channel the correct Portlet
Definition ID. If you don't you will get errors (not repeated here) telling you that uPortal couldn't find
that ID. Just make sure your ID is [webapp_name].[portlet_name] where [webapp_name] is the name of the webapp
that contains the portlet (same name as the .war) and [portlet_name] is defined in the portlet's portlet.xml file.
* And my portlet content did not show up, although I got no errors. After futzing around for a while, I realized I had made
the dumb mistake of omitting the call to the request dispatcher's include() method. Along the way, I managed to learn that
the portlet context is indeed for the webapp that runs it (not uPortal, but "testsuite" or whatever webapp), so the
requestdispatcher should be initialized with paths relative to the specific webapp. In other words, if you want to
load webapps/junk/junk.jsp, you use context.getRequestDispatcher("junk.jsp").
* After a couple of times recreating the war and redeploying it with uPortal's build target, I realized this procedure could
be shortened by just recompiling the portlet directly into the deployment location (ie the
tomcat webapp) and restarting the webserver to reload the classes rather than redeploying the portlet through uPortal
(which introduces uncertainties). This worked fine.
* System.out.println's indicate that the portlet runs doDispatch() after running init() and before doView()--more precisely, doView
is not called if doDispatch is there. If I remove doDispatch(), then doView() is called. Tra-la.
* My portlet definition xml file never shows up in uPortal's WEB-INF/classes/properties/chanpub directory. May have to
do this by hand, but I don't yet see that this is a problem.
* Another flakiness: if you delete a channel, it will still appear in the list of available channels until you log out.
This may be an update issue--if I wait long enough, the system configuration will update itself. But I did not follow up
on this--it is quicker to just logout and login.
* A useful debugging note: load your jsp pages from their deployment directories first to make sure they are debugged before
loading them through portlets. This only gets you so far, but at least it can catch simple compiler errors. Your portlet variables
(from the pluto definedObjects tag) will have null values, so the page won't work in standalone mode outside the container (ie by
loading directly in the browser).
* If you have mistakes in your JSP content (things that prevent it from compiling, noted above) you get
Channel ID: 94
Message: IChannelRenderer.completeRendering() threw
Error type: Channel failed to render (code 1)
Problem type:
Error message null [based on exception: null]
* One issue that will be important (and I think is undefined) is portlet caching rules. That is, how differently do different
containers handle caching? This is important for linked content that makes heavy use of forms. You don't want, for example,
to repost form parameters everytime you reload a tab. Likewise you don't want to lose state just by clicking on different tabs
before returning to your original portlet.
* Interestingly, JSP session variables and cookies loaded by portlets from different webapps have the same ID and the same
session cookie. But session attributes are different! Session attributes put into the session in one jsp page can be retrieved
by other JSP pages loaded/managed by the same portlet, but they can't be retrieved by JSP pages running in other webapps that
are managed by other portlets. Not unexpected, but it was unexpected that the session variables would think that they are the same.
* I'm taking this as the directory structure
src---build.xml [ant file for building, deploying, and warring the portlet
|
--external_libs [pluto/portlet jar files not to be included in the war]
|
--java [portlet source code]
|
--webapp [the deploy directory that gets warred up at the end]
The webapp directory has this structure:
webapp---images
|
--jsp [jsp pages loaded by the portlet]
|
--WEB-INF [classes, lib, web.xml, portlet.xml]
* If using ant, make sure you read the
structure it creates may not match your source directory--you have to specify lib and classes
explicily.
* I wasted a lot of time trying to deploy a simple portlet into uPortal and finally
found that my web.xml file had a problem. I originally copied this over from the
testsuite's web.xml file:
The uPortal tool "ant deployPortletApp" constantly gave "Exception in thread "main" java.lang.StackOverflowError".
This is in the Deployer.java file from uPortal, in the prepareWebArchive() method. The lines
WebApplicationUnmarshaller wau = new WebApplicationUnmarshaller();
wau.init(new FileInputStream(webXml), webModule);
were the source of the error--probably this wraps some castor-like thing that processes the xml. I don't know why it
failed for my test portlet but worked for the testsuite. Anyway, I changed this to
and it worked like a charm.
* I'm doing everything below as uPortal's "admin" user. I don't know or want to learn the uPortal channel control
permissions yet.
* After deploying the portlet, I can add it using the channel manager. Had problems because I did not read the
(uPortal portlet) instructions carefully as usual. You must make sure that you give the channel the correct Portlet
Definition ID. If you don't you will get errors (not repeated here) telling you that uPortal couldn't find
that ID. Just make sure your ID is [webapp_name].[portlet_name] where [webapp_name] is the name of the webapp
that contains the portlet (same name as the .war) and [portlet_name] is defined in the portlet's portlet.xml file.
* And my portlet content did not show up, although I got no errors. After futzing around for a while, I realized I had made
the dumb mistake of omitting the call to the request dispatcher's include() method. Along the way, I managed to learn that
the portlet context is indeed for the webapp that runs it (not uPortal, but "testsuite" or whatever webapp), so the
requestdispatcher should be initialized with paths relative to the specific webapp. In other words, if you want to
load webapps/junk/junk.jsp, you use context.getRequestDispatcher("junk.jsp").
* After a couple of times recreating the war and redeploying it with uPortal's build target, I realized this procedure could
be shortened by just recompiling the portlet directly into the deployment location (ie the
tomcat webapp) and restarting the webserver to reload the classes rather than redeploying the portlet through uPortal
(which introduces uncertainties). This worked fine.
* System.out.println's indicate that the portlet runs doDispatch() after running init() and before doView()--more precisely, doView
is not called if doDispatch is there. If I remove doDispatch(), then doView() is called. Tra-la.
* My portlet definition xml file never shows up in uPortal's WEB-INF/classes/properties/chanpub directory. May have to
do this by hand, but I don't yet see that this is a problem.
* Another flakiness: if you delete a channel, it will still appear in the list of available channels until you log out.
This may be an update issue--if I wait long enough, the system configuration will update itself. But I did not follow up
on this--it is quicker to just logout and login.
* A useful debugging note: load your jsp pages from their deployment directories first to make sure they are debugged before
loading them through portlets. This only gets you so far, but at least it can catch simple compiler errors. Your portlet variables
(from the pluto definedObjects tag) will have null values, so the page won't work in standalone mode outside the container (ie by
loading directly in the browser).
* If you have mistakes in your JSP content (things that prevent it from compiling, noted above) you get
Channel ID: 94
Message: IChannelRenderer.completeRendering() threw
Error type: Channel failed to render (code 1)
Problem type:
Error message null [based on exception: null]
* One issue that will be important (and I think is undefined) is portlet caching rules. That is, how differently do different
containers handle caching? This is important for linked content that makes heavy use of forms. You don't want, for example,
to repost form parameters everytime you reload a tab. Likewise you don't want to lose state just by clicking on different tabs
before returning to your original portlet.
* Interestingly, JSP session variables and cookies loaded by portlets from different webapps have the same ID and the same
session cookie. But session attributes are different! Session attributes put into the session in one jsp page can be retrieved
by other JSP pages loaded/managed by the same portlet, but they can't be retrieved by JSP pages running in other webapps that
are managed by other portlets. Not unexpected, but it was unexpected that the session variables would think that they are the same.
Subscribe to:
Posts (Atom)