Tuesday, December 30, 2008

Hibernate and EnhancerByCGLIB issues

Working with Hibernate can make performing routine database operations a breeze. Every now and then however you hit a snag that causes your brain to drop on the floor. I recently ran across one of these situations where objects were not being updated. After checking my syntax and inspecting objects, I realized the issue had to do with the fact that Hibernate will often create proxy objects as classes of type EnhancerByCGLIB when pulling collections that are marked as FetchType.LAZY. It also occurs when the SessionFactory is asked to load() the object instead of get().


First Case : Lazy Initialization Proxies:

The proxies may often contain all the information in your persistent class, and so they appear to be the real deal. When we query these objects they return all the information in the underlying proxied object. We run into a couple issues however if we use standard implementations of equals in our persistent class:

Lets say your equals() method in the class Noodle looks like this:

public boolean equals(Object obj) {
if(this == obj)
return true;
if(getClass() != obj.getClass() )
return false;
........
}


At this point we see that equals will never succeed - the class Noodle will never equal the Hibernate Proxy class. When you try to update an object from the collection, Hibernate never finds the object it needs to update because the equals implementation always fails. If you need to implement a comparison of class you could use one of the following :

this.getClass().isAssignableFrom(obj.getClass())

or

HibernateProxyHelper.getClassWithoutInitializingProxy(obj)

If you use hibernate annotations in your pojos, this additional Hibernate checking should not be too invasive or out of place. One other mistake people often make when implementing equals is to try to access the fields directly off of the object:


public boolean equals(Object obj) {
....
Noodle other = (Noodle)obj;
if(! this.attributeA.equals(other.attributeA) )
return false;
....
}


Here we run into 2 issues:

1. The cast may throw a ClassCastException.
2. The attributeA may have limited visibility.

A Solution that worked for me and covers the situation where the other object is not a proxy:

public boolean equals(Object other) {

if ( this == obj )
return true;

if( obj == null )
return false;

if( getClass().equals( obj.getClass() ) {
Noodle other = (Noodle)obj;
if( getId().equals( other.getId() )
return true;
}

else if( obj instanceof HibernateProxy ) {
if(HibernateProxyHelper.getClassWithoutInitializingProxy(obj).equals(this.getClass() ) {
Noodle other = (Noolde)obj;
if( getId().equals( other.getId() )
return true;
}
}

return false;
}

Second Case: You used Load() instead of get():

There may be times when you want to use load() - when you are simply adding references to an object in a collection for instance. But when you are pulling an object specifically to use its values, you should switch to using get() - as the attributes are guaranteed to be available.


Monday, December 8, 2008

Notes from a Windows to Mac Conversion Bender

I recently upgraded from my 2 year old pc to a new mac book pro. It was a tough decision because I need to be able to run oracle and the customer I am working with uses WebEx AIM Pro for desktop sharing. (I work remotely 75% of the time.) . The final decision to go mac was heavily influenced by the fact that I could use vmware to run an instance of my windows pc on the mac. The fear of Vista (Vistanoia?), the desire to leave the world of waiting for apps to respond, and the chance to try out a new toy were beckoning.

I pulled the trigger, and jumped on the configuration express.

I've been in config-hell before, and something deep down told me this would be no different. All the smiling faces that I met saying that changing to a mac was like a trip to disney surely had to have an unseemly downside lurking in a hidden log file.

The basic setup was pleasant - eclipse and tomcat installed as on a pc by simply unzipping them into the directory where they would be run from:

%> sudo gunzip apache-tomcat-5.5
%>sudo tar -xvf apache-tomcat-5.5
%>cd apache-tomcat-5.5/bin
%>./startup.sh (You are now running tomcat)

Eclipse was similiar.

One thing to get used to on a mac is that you no longer download jdks from sun. The jdk is preinstalled on your machine. You go to the Apple site and download the jdk for the mac if you need a different version or need an update. Apple doesnt seem to supply options for minor versions.

I ordered up a copy of VMWare fusion and planned to install my old pc on the mac as a resource - the ability to access old email, documents etc without having to go back to an old machine. However I had overlooked one issue - the windows xp had been installed on my pc when I bought it, and when I booted it up on the vmware, windows insisted on an activation key. The old one on the laptop was no longer valid, and so I had three days of use before the activation grace period expired. In that time I copied files over to the mac and made the most of the situation. The VMWare fusion worked greeat - and it was phenomenal to see the old pc running on the mac.

I would suggest taking snapshots of your OS in vmware, and make sure you keep the original so that you can always return to it. After three days however, I said goodbye to my old pc on the mac.

The next step for me was to get Oracle installed on the Mac. This was simple enough by installing Ubuntu on a vm in the vmware fusion. Again, vmware worked like a charm and Ubuntu is a very nice and user friendly. I used Ubuntu 8.10 and downloaded an OracleXE database. Everything went smoothly, and the database runs without issue. With vmware I can actually drag and drop files between desktops, and the database has been running without issue.


On Ubuntu, before you start running all over the net looking for apps or jdks to download, use the Synaptic Package Manager under the System tab on your desktop. Real simple.

The stumbling block for me was that the project I work on uses sqlldr to load the database. This executable processes a flat file and inserts the data in the oracle database based on a template definition. I wanted to run the eclipse and tomcat on the mac, and the oracle instance on the Ubuntu vm. I read that installing the oracle client on the mac was possible, but when I did, I found that it did not come with the sqlldr utility.

My first plan to circumvent the issue ws to run eclipse and tomcat on the Ubuntu instance. This worked, but eclipse had issues - perspectives needed to be configured with each new restart, and I found eclipse crashed in the vm frequently. I needed an environment that was stable and able to handle heavy loads on the processor and manage the memory well. This was not the case on Ubuntu in the vmware virtual machine.

The next idea was to install A Samba server on Ubuntu. Samba allows you to access other machines, in this case the virtual machine on my mac. The install call was

%> sudo apt-get install samba smbfs

This allowed me to access files across machines - however I was not able to execute the sqlldr command from the mac on the Ubuntu machine.

My next plan of attack is to update to the OS/X Server and install the Oracle Client for 10g.

And so it goes...









Monday, December 1, 2008

Tomcat and Eclipse on Ubuntu

The following is a quick review of the steps I took to get eclipse and tomcat up and running and playing well together. Replace "user" with your local user name. I probably opened things up a bit wide security wise. It works. 
  • Download the tar.gz files for linux from the respective web sites. 
  • gunzip the files 
  • tar xvf the files. 
  • Move Tomcat to /usr/share/tomcat
  • sudo chown -R user:user tomcat
  • sudo chmod -R a+rwx tomcat
  • sudo chmod +x `sudo find tomcat -type d`
  • Move eclipse to /opt
  • sudo chown -R  user:user eclipse
  • sudo chmod -R a+rwx eclipse
  • sudo chmod +x `sudo find eclipse  -type d`
The net affect of the above is that the applications are deployed to their respective locations and the user "user" may run them. I was able to run each separately, but was having issues with eclipse trying to run tomcat. These errors were due restrictions in writing logs and deploying to tomcat directories due to permission issues. The above commands allow all users to write to these directories. 

Next, you can create an executable that specifies an ECLIPSE+HOME variable: 
Create an executable for eclipse in usr/bin called eclipse
  • #!/bin/sh
  • export ECLIPSE_HOME="/opt/eclipse"
  • $ECLIPSE_HOME/ECLIPSE $*
Save the file and run clean: 
/opt/eclipse/eclipse -clean

Add the two applications to the menu via System>Preferences>Main Menu. 

In eclipse, add Tomcat as a server, and in the tomcat configuration, under "server Locations", choose "Use Tomcat Installation" radio button and save. 



SQLDeveloper on Ubuntu

To Install SQLDeveloper, download the application from http://www.oracle.com/technology/software/products/sql/index.html. Unzip the archive and move it to the directory of choice. I chose /usr/share. Change the file to be executable: 

  • unzip sqldeveloper.zip
  • sudo mv sqldeveloper /usr/share/sqldeveloper
  • sudo chmod +x /usr/share/sqldeveloper/sqldeveloper.sh 
Then invoke the sqldeveloper.sh : 
  • sh sqldeveloper.sh
You will be asked to specify the path to the java home. You are up and running. 

To add SQLDeveloper to your menu, Go to System?Preferences>Main Menu. Add SQLDeveloper to the menu of choice. The Application is now executable from your menu the next time you log in. 

Install Samba on Ubuntu

I have a vm on my Mac running as a guest with Ubuntu as the OS. In order to access files on the vm guest, I added Samba - a file and print server. To install Samba,  and edit the configuration: 
  • sudo apt-get install samba smbfs (Installs Samba)
  • sudo gedit /etc/samba/smb.conf (Edits the config file)
Then, go to the "Share Definitions" section of the config file that is open in your editor: 

find the lines:
  • ; [homes]
  • ; comment = Home Directories
  • ; browseable = yes

Remove the ";" to uncomment these. This allows you to access the home directories on the guest.  Then find the following line that specifies the read only attribute. It is set for yes by default. To be able to write to your directories, change this to "no" and uncomment it. 
 
  • read only=no  (if you want to be able to write to the directory)

Next you need to find the line "; security = user". Replace it with the following 2 lines: 
  • security = user
  • username map = /etc/samba/smbusers
This specifies that users may authenticate, and that the users are specified in the /etc/samba/smbusers file.  Next we need to add entries to the users file. 
  • sudo gedit /etc/samba/smbusers
The file will be empty. If you want to access your local profile, and lets say the login name is Gus, then add the following line to smbusers: 

  • Gus="Gus" 
Save the file and from the command line create a Samba password: 
  • sudo smbpasswd -a Gus
You will be prompted to enter and confirm the password for this user. Thats it , you are done with the install of Samba. 


To stop | start | restart Samba: 
  • /etc/init.d/samba stop | start | restart
The config file allows you to change the behaviour of the samba server. There are comments in the file that point you to resources for evolving the file and perhaps opening up you system to domains. 

To Access the samba server, go to : 
  • smb://computer_name/Gus
On a Mac, this is under Finder>Go>Connect to Server>.  You will be prompted for a authentication with the password you specified to smbpasswd. If authentication works, the disk will mount on your desktop, and you will be able to browse within your profile on the remote machine. 


Sunday, November 30, 2008

Install Java on Ubuntu

I thought this would be straight forward....

I tried to install using the apt-get command, but my install died when the license agreement took over my terminal, and did not give me a way to accept the agreement. I exited and ran the dpkg --configure -a command to clean up the failed install. 

Instead I went to the System>Synaptic Package Manager and performed the install this way: 
  • Specify "java" in the search box. A List of packages to install appears.
  • I chose the java5 jre, jdk and plugin, and accepted the installation of packages that they were dependent on. 
  • Click Apply
  • Accept the license agreements as they appear.
  • The java home directory will be in /usr/lib/jvm
Note: If the java versions do not appear in the above list, ensure that the /etc/apt/sources.list file contains the following entries : 

  • deb http://us.archive.ubuntu.com/ubuntu/ intrepid multiverse
  • deb-src http://us.archive.ubuntu.com/ubuntu/ intrepid multiverse
  • deb http://us.archive.ubuntu.com/ubuntu/ intrepid-updates multiverse
  • deb-src http://us.archive.ubuntu.com/ubuntu/ intrepid-updates multiverse

You then need to ensure that this Java is the default java for the system: 
  • update-java-alternatives -s java-5-sun (specifies java 5 as the default)
  • update-java-alternatives -l (lists out the default)

Finally, add environmental variables. I add them in /etc/environment. You could also add them in your local .bashrc file in your home directory. The values specified in /etc/environment are in all shell. Be careful of the syntax in this file! If an error occurs you may not be able to boot up... : 

  • JAVA_HOME="/usr/lib/jvm/java-1.5.0-sun"
  • CLASSPATH="/usr/lib/jvm/java-1.5.0-sun/bin:/usr/lib/jvm/java-1.5.0-sun/lib"




Install Oracle XE on Ubuntu

Oracle XE can be downloaded at the Oracle site. I chose the Debian install for Linux. I read the installation guides. This is how to do it:

If you do not have 1GB free memory, or want to reduce the database footprint:

  • $sudo dd if=/dev/zero of=/swpfs1 bs=1M count=1000
  • $sudo mkswap /swpfs1
  • $sudo swapon /swpfs1
I used the Ubuntu apt-get utility to install the database.
  • sudo gedit /etc/apt/sources.list
Insert "deb http://oss.oracle.com/debian unstable main non-free" into the file if it does not exist and save. Then import the Key:
  • $wget http://oss.oracle.com/el4/RPM-GPG-KEY-oracle -O- |sudo apt-key add -
Now install the db:
  • $sudo apt-get update
  • $sudo apt-get install oracle-xe
After the installation you need to run the following:
  • sudo /etc/init.d/oracle-xe configure
Follow the instructions and write down the ports and passwords that you specify. Then you will need to update the users on the system. Go to System>Administration>Users and Groups. Unlock the editor and under the manage groups tab, add your user to the dba group.

The database should be running. Go to http://localhost:8080 - assuming you didn't change this port in post config, and the database info page should appear. The database is now integrated in the menu as well - so if not running, start it it up by navigating to the Applications>Oracle Database>Start Database.

One more step is to specify the ORACLE_HOME and ORACLE_SID environmental variables:
  • gedit $HOME/.bashrc
Add the following lines:
  • ORACLE_HOME=/usr/lib/oracle/xe/app/oracle/product/10.2.0/server
  • ORACLE_SID=XE
  • export ORACLE_HOME ORACLE_SID
You could also/instead add these to your /etc/environment file. In that file you would add:
  • ORACLE_HOME="/usr/lib/oracle/xe/app/oracle/product/10.2.0/server"
  • ORACLE_SID="XE"
Either login in again, or type
  • source .bashrc
  • source /etc/environment
To run Oracle utilities add /usr/lib/oracle/xe/app/oracle/product/10.2.0/server/bin to the path in /etc/environment and source it.


You should be good to go. You can access the database home page via the menu, or go to http://localhost:8080/apex.





Install VMWare Fusion on Mac OS/X 10.5

I needed to run an oracle database on a macbook. At the time, Oracle was not supported on Mac OS/X 10.5 . Talking to colleagues, and doing some research led me to VMWare fusion. It's not free. It works great.

Installing VMWare Fusion is a relatively simple process. I downloaded version 2.0.1-12 from the net and purchased a serial number from VMWare. The VMware site has excellent documentation and even supplies videos to walk you through the process. Here's what I did: 
  1. Download the .dmg file from vmware. (http://www.vmware.com/vmwarestore/buyfusion.html).
  2.  Double click on the .dmg file to mount it. The file will be mounted and displayed in a Finder window.
  3. Double click the Fusion Icon. The installation assistant will walk you through the process. 
  4.  To Start Fusion, navigate to Applications in Finder, and choose the VMWare Fusion Icon. 

The Fusion window will open, and you can create a new Virtual Machine at this point. Next I installed 2 different virtual machines - a copy of my old pc and a fresh Ubuntu install to house the Oracle Database. 

I followed the conversion guide for bringing my old pc over. The guide that comes with fusion (http://www.vmware.com/pdf/fusion_getting_started_11.pdf) has a link to a flash video that walks you through the process. The process took about 8 hours for the vmware converter to bundle my old pc onto an external disk drive and for me to boot the pc on the mac. It was pretty cool. The problem that I had was that my copy of XP only had 1 license, (it came bundled on the pc when I bought it) , and Windows required a new activation key. So I only had access to my old pc on the mac for 3 days. I would not do this unless you know you have multiple licenses. 

To install Ubuntu, I went to the Ubuntu site (http://www.Ubuntu.com/getubuntu/download) and downloaded the 8.10 version on 32 bit. I chose the 32 bit because the 64 bit has some issues with downloading and installing packages. Once downloaded, simply open the vmware fusion :


  1. Choose new 
  2. Click the continue without disk button. 
  3. Choose the "Use Operating System Disk image file" option.
  4. Choose the Ubuntu file you downloaded above from the finder that opens. 
  5. Click Continue and Finish. The new virtual machine will appear in the Fusion window. 
  6. Click the start button next to the Ubuntu vm, and fusion will open a window with the Ubuntu installation running within it. 



I ran through the installation by specifying users and some configuration. I simply accepted the defaults. 

Sage Advice
Install VMWare tools immediately after the Ubuntu installation occurs. Choose Install VMWare tools from the virtual machine menu item on the mac. To do this, move the untarred vmware tools distib to the /tmp directory.  Then run : 
  • sudo ./vmware-install.pl
Choose the defaults, and when complete, add the toolbox to the startup of your Ubuntu session: 

  1. Preferences>Sessions>StartUp Programs>+Add>
  2. Specify vmware-toolbox as the command to execute. 

Fusion allows you to take snapshots of your vm. DO THIS!!!! . Comment well the snapshot state, so that when you screw something up you can return to a stable state.  As I was installing the various applications to the Ubuntu system , I would stop and take a snapshot along the way after each milestone. This will save you considerable time. 




Thursday, November 27, 2008

eclipse on mac

I run eclipse against an oracle instance with htmlUnit tests banging away on tomcat. Running the complete suite of unit tests on my dev box can eat processor and memory big time. Its like watching a kid coming home from college for the first time in months and invading the fridge. On a pc there was no processing time left for me. When I set up my env on the mac, and ran with the base settings, it died by running out of heap space.

So I try to allocate enough memory and set the parameters on the various running jvms (ant, tomcat and eclipse) in the hope that they wont die after running for 50 minutes with a heap space space issue. My box has 4G of memory, and I have oracle running in a separate vmware fusion vm on Ubuntu. I am using eclipse Ganymede with java 5 (1.5.0.13) , Tomcat 5.5 and and use ANT4.7.

On Mac, the eclipse.ini file is under your eclipse installation at /Eclipse.app/Contents/MacOS/eclipse.ini . You will need a terminal window to get there as the directory doesn't show up in Finder. You could perform the following to find it:

sudo find / -name "eclipse.ini"

When I opened this for the first time I found that the parameters were triplicated - so I cleaned this up. I jack up the max memory, specify to use parallel GC, and keep the eden space low. I specify the following arguments:

-Xmx1024m (the max heap size)
-Xms128m (Startup heap size)
-Xmn64m (the eden space or young generation garbage collectible space)
-XX:+UseParallelGC (Run Parallel GC)


In java there are two GC threads running. One is a very lightweight thread which does incremental collections primarily on the Eden (a.k.a. Young) generation of the heap. The other is the Full GC thread which traverses the entire heap when there is not enough memory left to allocate space for objects which get promoted from the Eden to the older generation(s). If there is a memory leak or inadequate heap allocated, eventually the older generation will start to run out of room causing the Full GC thread to run continuously. This GC thread eats the processor and eventually eclipse has no pie - it won't be able to respond to requests and they'll start to back up.


The amount allocated for the Eden generation is the value specified with -Xmn. The amount allocated for the older generation is the value of -Xmx minus the -Xmn. Generally, you don't want the Eden to be too big or it will take too long for the GC to look through it for space that can be reclaimed. If you watch the memory monitor closely, you can see the garbage collector releasing memory periodically. Keeping the amount allocated to the eden space low affects the periodicity of this cleanup. Developers can see this on their machines when running the complete stack of tests.

These parameters seemed to work for me- my memory consumption remained nearly constant across the task. Before, the memory usage would creep up and reach a terminal point - on my old pc at about 2.4GB. At this point the tests just stopped running. Viewing the memory usage after applying the new args, you could see where GC ran periodically and brought the usage back to a baseline. The tests ran faster. But there was still an occasional lockup.

Another consideration is whether to run ANT, Tomcat and Eclipse in the same jvm. I tried experimenting with combinations of virtual machine with these processes. I also experimented with the memory allocations for each. What I found was that if you run them all in the same jvm, then the memory allocation on your machine remains high even after the tasks complete. When you run these in separate jvms, the memory allocation is returned to the system as each process completes. I found that the machine performed best when running separate jvms.


Caveat Emptor: Please be aware that not all jvms support the run parallel GC option.

To specify the jvm parameters for ANT, go to the ANT view, right click on the build file or a task, click 'Run As' and then 'external tools configuration'. In the dialog that appears, choose the JRE tab and specify the vmargs and the jvm to use for the file/task.

To Specify the jvm parameters for Tomcat, double click on the tomcat instance in the servers view. Update the vmargs by clicking the Launch Configuration link.