Faster GWT startup with objects embedded in the HTML host page

I recently ran into an interesting article about dumping some data into the HTML host page for your GWT app to reduce the number of round trips to the server. Since my app loads a lot of little configuration items from the server on startup, I thought this would be an interesting technique and decided to make a good type-safe implementation of it by extending my DictionaryConstants class in Kiyaa!

The results? I can now write arbitrary serialized objects into my host HTML page using DictionaryConstantsWriter and load them in a type-safe manner using a DictionaryConstants instance. It’s quite nice a convenient, check it out in the Kiyaa! source tree.

Bookmark and Share

Filed under: GWT, java, web development | Posted on February 17th, 2009 by Dobes | Comments

How to workaround JPA locking limitations when using Hibernate by bypassing the EntityManager and using Session directly

First, the solution:

    private Session getSession() {
        return ((Session)((EntityManagerImpl)em.getDelegate()).getSession());
    }

Now you can call the methods on Session which handle locking using LockMode.

A brief introduction to JPA’s optimistic locking system as I understand it

For performance reasons, the designers of JPA decided that optimistic locking was the way to go, and did not include any pessimistic locking capability to the EntityManager API (that I know of). Concurrency is handled using a “retry” system - if during a transaction another thread changes an object you are also changing, you are told to retry. This message arrives in the form of an OptimisticLockException, and after that exception is thrown your transaction is rolled back and thrown away never to be used again.

If the user is manually making some edits and another user or automatic process edits the same objects, the developer is expected to catch the OptimisticLockException and, if it is not possible to automatically merge the changes, present the user with a means to resolve the conflict. The code can get its hands on the “new” version by fetching it from the database, and use its current “detached” entity as the “workspace” revision.

Automated processes follow a similar process, except they have to decide in an automatic fashion how to deal with the concurrent change. Usually this means just re-starting the operation.

What’s wrong with JPA locking anyway?

The optimistic locking system relies on the ability to fully roll back an operation. This works great if everything affected by the operation participates in transaction, but it is completely useless if your code has side-effects that do not participate in transactions, such as sending email, accessing (non-transaction-supporting) third-party web services, or charging credit cards.

The only workaround I could think of using the EntityManager API as it is, is to make sure that you do all the non-transactional operations after the all the transactional ones are done. Try to do only one per transaction, since if that operation throws an exception your transaction could get killed and you’ll have no standard way to store which operations completed successfully.

I have a hunch that some use of @TransactionAttribute could be partly helpful with this but I’m still a bit shaky on the use of that annotation.

Hibernate EntityManager’s Pessimistic Locking Doesn’t Help

I’d read that calling EntityManager.lock() when using Hibernate EntityManager would issue a select … for update and thought that this would solve my problem. However, consider a piece code like:

X x = em.find(X.class, id); // SQL SELECT from the database
em.lock(x, LockModeType.READ); // SQL SELECT ... FOR UPDATE WHERE version = :version

There is a race condition between the find() and lock() that cannot be resolved, since lock() validates the version number in the object that it is given. So yes, this will wait for the other transaction to complete and then take a mutually exclusive lock. However, the next thing it’ll do is throw OptimisticLockException if the other transaction changed the object, which defeats the point.

Hibernate Session’s Pessimistic Locking

Use the code at the beginning of this article you can access the Session object in Hibernate directly, while still using the EntityManager. This is nice because you get all the power of a hibernate Session with the convenience of dependency injection and container-managed transaction.

Now to fetch and lock an object, you can use code like:

return (X)getSession().get(X.class, (Serializable) id, LockMode.UPGRADE);

There is no race condition because the object is fetched freshly from the database after any other transaction using the object is complete. There is no version number check and you cannot get hit with an OptimisticLockException. This method does assume you are fetching the object by ID, however.

You can also upgrade an entity you already have to a locked state like this:

        Session session = getSession();
        LockMode lockMode = session.getCurrentLockMode(entity);
        if(lockMode.lessThan(LockMode.UPGRADE))
            session.refresh(entity, LockMode.UPGRADE);

Note that if the entity was not already locked when this code runs, it will be refreshed from the database (all changes already made will be discarded). This should be fine since you shouldn’t have made changes to the object without locking it anyway. If we did not use refresh() then the version number might be out of date and we’d get an OptimisticLockException when the transaction completed.

Bookmark and Share

Filed under: eclipse, glassfish, java ee | Posted on February 7th, 2009 by Dobes | Comments

Running Work in a Seperate Thread in Glassfish using the WorkManager

While working on our business accounting software, I recently learned about a very useful class in glassfish from a friendly fellow on the glassfish users mailing list. This is the WorkManager, a class that allows you to add Runnable bits of Work for glassfish to run in another thread pool.

I use this to perform independent tasks asynchronously. For example:

        try {
            String threadPoolName = "backend-jobs"; // create this thread pool in the admin panel or using asadmin
            WorkManager workManager = WorkManagerFactory.getWorkManager(threadPoolName);
            Work work = new Work() {
                @Override
                public void release() {
                    // glassfish has requested the job to stop ...
                }
 
                @Override
                public void run() {
                    // Gets run in a thread
                    // UserTransaction can be used here to start, commit, rollback a transaction if you need database access
                }
            };
            workManager.scheduleWork(work);
        } catch (WorkException e) {
            throw new Error(e);
        } catch (ConnectorRuntimeException e) {
            throw new Error(e);
        }

I had previously used the EJB TimerService to achieve this but I think that this approach is probably better, for those jobs that I want to run semi-immediately.

Notes:

  • If you look at the WorkManager API you will see that you also have the option of waiting for work to start, or waiting for work to complete, which makes this much more flexible than using the TimerService.
  • You specify the name of a ThreadPool when calling WorkManagerFactory.getWorkManager(); this ThreadPool should be created in the application server before deploying your application
  • You may specify null for the thread pool name to use the default ThreadPool
  • Use a SessionContext object and call getBusinessObject() to get a proxy to your EJB which will construct the needed context to make all the EJB “stuff” work correctly if you call any method on it (it’s the same type of proxy used when you inject the EJB into another object).
  • You can use the UserTransaction class to create a database transaction in the run() method which can be used to access the database.
Bookmark and Share

Filed under: glassfish, java, java ee | Posted on February 3rd, 2009 by Dobes | Comments

How to parse XHTML faster by not loading DTDs from w3c.org

IMG_4581If you are reading this, you may be running into the same issues I did when parsing XHTML files, namely:

  • Slow parsing of XHTML
  • java.net.ConnectException: Connection refused exception when parsing XHTML code

The cause of these errors is that the XML parser is loading the XHTML DTDs from w3c.org in order to resolve named “entities” like   and © so that it can convert those into java strings correctly.  Sadly, although the Java libraries are smart enough to go fetch these DTDs, they currently do not support or participate in any kind of HTTP client side caching.  Thus, if you are doing a lot of XHTML parsing you will be talking to the W3C servers a lot, which sucks.

Although solutions to this problem is documented in various places on the internet, including:

They don’t really provide a piece of simple source code, or even better a jar file, that you can drop into your application and use immediately.  Being the social activist that I am, I decided to make one so that future XHTML parsers won’t have to suffer as I did.

Check it out on Google Code here: java-xhtml-cache-dtds-entityresolver

Bookmark and Share

Filed under: html, java, xml | Posted on January 4th, 2009 by Dobes | Comments

How to use EJB Timers via the TimerService

©1996 Image Club Graphics A division of Adobe Systems Incorporated

©1996 Image Club Graphics A division of Adobe Systems Incorporated

Java EE provides a the TimerService to allow you to schedule actions to run at a future time.  The ability to schedule timed jobs, or just delay a piece of work so it runs asynchronously, is an important part of most software systems.  When I first read about the TimerService I was a bit confused by the way that it works and initially I avoided uses it because I didn’t understand it.  However, now that I have a clearer idea of how to use it, I’m quite pleased with it.

First of all, the features of the TimerService:

  • Each bean gets its own TimerService and its own list of timers
  • Timers are stored in the application container’s internal database and survive shutdown, restart, and redeployment of the application
  • When timers are fired, they usually create an EJB Transaction so you can access the database (Entity beans)

How do I use it?

The timer service is injected using annotations, like the EntityManager and EJBs:

    @Resource
    TimerService timerService;

To schedule a future event, use the createTimer() method, for example:

    public void initTimers() {
        timerService.createTimer(5*1000, null); // Runs ~ 5 seconds from now
    }

Looking at that code, you’ll see that we’ve hit the first piece of “weirdness” with TimerService - you don’t specify what to do when the timer fires! Luckily, the Java EE Tutorial is kind enough to show us this much information; by annotating a method with @Timeout we indicate that this method should be called when any timer for that bean’s injected TimerService fires. In point form:

  • The method annotation @Timeout is invoked when the timer fires
  • The same method is used for all timers on the bean
  • The bean whose timeout method to call is determined based on which bean the TimerService was injected into

Example:

    @Timeout
    public void timerFired(Timer timer) {
        System.out.println("Timer fired!");
    }

This is basically where the Java EE Tutorial leaves you - I had more questions.

What if I have multiple timers?

Many beans will want to have multiple timers running at once, obviously. The TimerService supports this - and quite well, in fact - using the info field of the timer. When you call createTimer() you pass in the info value as the last parameter. This is a Serializable object which is stored to the database along with the timer and you can retrieve it in your timeout method using the getInfo() method.
Here’s one example where we have different types of events we want to handle, but the events don’t contain any custom data:

   enum TimedEvent {
       EVERY_10_SECONDS(10*1000, 0),
       HOURLY(3600*1000, 0), 
       DAILY(24*3600*1000, 4*3600*1000), // Add a few hours to avoid DST issues
       WEEKLY(7*24*3600*1000, 4*3600*1000),
       EVERY_30_DAYS(30*24*3600*1000, 4*3600*1000),
       MONTHLY(1000,0);
       final long interval;
       final long offset;
       private TimedEvent(long interval, long offset) { this.interval = interval; this.offset = offset;}
       public long getInterval() { return interval; }
       public long getOffset() { return offset; }
   }
   public void scheduleTimers() {
       long now = System.currentTimeMillis();
       for(TimedEvent te: TimedEvent.values()) {
           if(te == TimedEvent.MONTHLY) {
                Calendar cal = Calendar.getInstance();
                cal.clear(Calendar.MINUTE); cal.clear(Calendar.SECOND);
                cal.set(Calendar.HOUR, 4);
                cal.set(Calendar.DAY_OF_MONTH, 1);
                cal.add(Calendar.MONTH, 1);
                timerService.schedule(cal.getTime(), te);
           } else {
                timerService.schedule(now - (now % te.interval) + offset, te.interval, te);
           }
       }
   }
   @Timeout
   public void timerFired(Timer timer) {
       TimedEvent evt = (TimedEvent)timer.getInfo();
       switch(evt) {
           case EVERY_10_SECONDS: // do stuff we do every 10 seconds
           case HOURLY: // do stuff we do every hour
           case DAILY: // do stuff we do every day at midnight
           case WEEKLY: // do stuff we do every week
           case EVERY_30_DAYS: // do stuff we do every 30 days
           case MONTHLY: // Have to manually schedule the next one, months are not fixed length
                Calendar cal = Calendar.getInstance();
                cal.clear(Calendar.MINUTE); cal.clear(Calendar.SECOND);
                cal.set(Calendar.HOUR, 4);
                cal.set(Calendar.DAY_OF_MONTH, 1);
                cal.add(Calendar.MONTH, 1);
                timerService.schedule(cal.getTime(), evt);
                // do monthly stuff ...
                break;
       }
   }

Remember that due to daylight savings time, you should avoid scheduling things at midnight in your current timezone, because they may end up being at 11pm the previous day when the time changes which might give unexpected results.

How do I start the timers?

One important element missing from the above is: who is going to call scheduleTimers()?
This does seem to be a bit of a gap in the Java EE spec; they don’t provide an obvious way to run a method when your EJB Jar is loaded (I haven’t found one).
I’ve seen examples that use the @PostConstruct annotation on a method, like this:

   @PostConstruct
   public void scheduleTimers() {
         // ...
   }

This would cause scheduleTimers() to be called each time an instance of your session bean is created before the first interface method is called on it. I haven’t tried this method because I only recently encountered it, and although it seems promising there are a couple of things I don’t know about it:

  • Would it be called several times on a stateless session bean if the container decided to create multiple instances of that bean?
  • For a stateless session bean, will it be called if no methods are ever called?

The method I’ve been using with some success is to create a ServletContextListener which invokes the scheduleTimers() method, like this:

public class MyBeanContextListener implements ServletContextListener {
    Logger logger = LoggerFactory.getLogger(getClass());
 
	@EJB
	MyBean bean;
 
	@Override
	public void contextDestroyed(ServletContextEvent event) {
            // TODO Should we cancel timers here?
	}
 
	@Override
	public void contextInitialized(ServletContextEvent event) {
		try {
  		    bean.scheduleTimers()
		} catch(Exception x) {
		    logger.error("Failed to schedule timers", x);
		}
	}

This has to be registered in the web.xml for a WAR in the same EAR, like this:

  <listener>
    <listener-class>MyBeanContextListener</listener-class>
  </listener>

It will be run when the WAR file is loaded or if the application server is restarted. However, this does mean that if you restart your application server, you may end up scheduling this method twice! Which brings us to…

How do avoid scheduleing the same timer twice?

I was previously burned because over time my timers had been scheduled multiple times, so I had the same code running twice. This would have been OK, except that code wasn’t using locking correctly. Oops!
Anyway, now you can learn from my mistakes. Before scheduling new timers, you should check for and cancel any old timers that are no longer relevant. For example:

    public void scheduleTimers() {
        for(Object timerObj : timerService.getTimers()) { // No clue why getTimers() doesn't have at parameterized collection
            Timer timer = (Timer)timerObj;
            if(timer.getInfo() instanceof TimedEvent) // Cancel all our recurring events
                timer.cancel();
        }
        // Now (re-)schedule the timers again
        // As an alternative, you could detect the timers that are already scheduled and not re-schedule them.
    }

What sorts of cool things can I put into the info field of a Timer?

When I first used the TimerService I used a String for the info field of the timer, like in the example in the Java EE Tutorial. The example never even used that string, which made the example a bit puzzling - I initially thought the string was there to identify the timer elsewhere, like in the monitoring system. Not so! A String is about the least useful thing you can use as the info field of your Timer - you can put any Serializable object there.
In one of my session beans I created an interface called Job with a single method perform(MyBean bean). By creating Serializable objects that implement this interface and call back to the bean with the appropriate method, I could schedule various different operations at different times, including some parameters to those objects, which are stored as members of the Job implementation.
You could also put an Entity Bean into the info field if it is Serializable, although if it has any lazy collections or non-transient references to non-serializable objects it wouldn’t work. I’d suggest putting the primary key of your entity instead and look it up when the timer fires. This will probably have fewer issues to deal with.

Will a timer survive redeployment of my enterprise application?

It’s pretty clear from the documentation that you can shut down the application server and start it up again and your timers will still fire. It will also notice any timers that should have fired when the application server was not running and fire them on startup. Also, clearly any timers you cancel will not fire again.
However, the timers for an EAR will be cancelled if the EAR is undeployed or redeployed. For this reason you should always be prepared to reinstate timers that could have been cancelled by a deployment operation, and not duplicate timers which may still be around because the application server has merely been restarted. This also means you can’t assume your timer will necessarily fire, since your application could be undeployed or redeployed before the timer could be fired. You will need an algorithm to reinstate or reinforce a timer’s work in case it is cancelled by the container.

What are some good uses for timers?

  • Sending emails It can be useful to send an email in a separate transaction/thread than the one that causes it to be sent. If email sending fails, you can just re-schedule the email to try again later, which can make your email delivery more reliable.
  • Asynchronous one-way operations - some operations on a bean aren’t supposed to keep the caller waiting, or even interfere with their transaction by throwing an exception and triggering an unwanted rollback. By scheduling a timer with a zero timeout I can defer that work. Examples: sending email (as above), cleaning up temporary files, …
  • Break down long-running operations - if you have a long-running operation iterating over many objects, which doesn’t necessarily have to run everything in one transaction, you can have an object storing the “working state” of that operation and have it run one chunk, then schedule itself to run the next chunk, and so on until it is complete. This avoids a heavy memory footprint that could be caused by loading all those objects into the session. In theory you can also use EntityManager.clear() to reduce the size of your session but in practice I’ve found this method may trigger various bits of poorly-tested code, lazy initialization exceptions, and more.
Bookmark and Share

Filed under: glassfish, java, java ee, servlets, web development | Posted on December 14th, 2008 by Dobes | Comments

Announcing Kiyaa! - a GWT Library for Accessibility, Styling, and More!

heeee-yaKiyaa! is a GWT toolkit for better accessibility, styling, and more!  It was created in the process of making small business accounting software that I needed to be easy to use, attractive, and to maintain.

It’s released under the MPL, so you can use it in your commercial applications and copy the code into other libraries or software if you want to.  Here are some highlights:

  • XHTML Template System based on facelets (also an example for how to write a GWT Generator)
  • Utility classes for sharing, combining, and proxying asynchronous callbacks
  • Server-side support for GWT’s statically typed localization (Constants and Messages) using javassist
  • Caching utility classes to help write client-side caches for RPC calls
  • A calendar widget supporting the jscalendar skins (originally a wrapper but I re-wrote it in Java)
  • A ComboBox class with search/suggest support and actions
  • A wrapper for the excellent DateJs date parsing library

It’s a very “fresh” release and I haven’t put any time into examples or documentation yet, so be prepared to learn it from the source code, and to have to submit patches and documentation of your own to get full value out of it.  Not for the faint of heart!

Link: Kiyaa! GWT Toolkit on Google Code

Enjoy!

Bookmark and Share

Filed under: Uncategorized | Posted on November 9th, 2008 by Dobes | Comments

JPA Locking and Timers bit me in the arse!

Accidentally double-charged a bunch of credit cards this morning, had to clean up a big mess.

Lessons:

  • Learn about how JPA locking REALLY works, I thought I was getting serializable transaction isolation, in fact it’s optimistic locking. Hibernate will generate an exclusive write lock on an object row if you use em.lock() on it, other providers might just throw an exception when you commit transaction (which may be after you’ve already charged the credit card, sent an email, etc. luckily I am using hibernate).
  • Learn how the timers system really works. I wouldn’t have bumped into item #1 if I hadn’t accidentally ended up with two timers running at the same time.
    Before adding your new timers, cancel the old ones: 

    public void cancelJobs() {
        for(Object timerObj : timerService.getTimers()) {
            Timer timer = (Timer)timerObj;
            // Assuming the class name is used as the timer info
            if(getClass().getName().equals(timer.getInfo())) timer.cancel(); 
        } 
    }

I’m glad I learned this lesson early enough that the mess was still manageable; even if you are not programming in Java I suggest you take this chance to review your locking strategy and make sure it’s working the way you expect it to.

Bookmark and Share

Filed under: database, hibernate, java ee, web development | Posted on November 1st, 2008 by Dobes | Comments

Hibernate Annotations: @OneToOne with mappedBy and lazy fetch are dangerous!

Today I ran into a very frustrating issue with hibernate and lazy inverse @OneToOne mappings. Inverse @OneToOne mappings are mapping where you provide the mappedBy attribute to indicate that the foreign key of that relation is stored in the other table.

I was using this technique as a lazy (as in, me trying to avoid work) way to access objects on the other side of certain relationships. For example, I have a CollaborationContext object which manages tagging, conversation threading, and file attachments for my online accounting software. This same class is referenced from several other classes (using @OneToOne), and occasionally I might have one of these and want to know from which of those objects own this object.

For whatever reason, I had thought that Hibernate would override my getters and setters and do the lazy loading that way, running a select on the database when the getter is called. OpenJPA, for example, does this (and reports all kinds of validation errors if you access fields directly). Whatever magic Hibernate would need in order to do this, it doesn’t.

Therefore, in order to support an inverse @OneToOne of this type, hibernate must fetch the related object each time the collaboration context is fetched, in order to check whether or not the other object should be null. Otherwise your program logic could be broken, since you’d either always have a null, or always have a non-null value.

The result, for me, was a ton of extra queries and really poor performance. Over the weekend I had added a ton of these lazy relations without realizing their negative performance impact. Now I’m deleting those relations and running an HQL query to manually handle that relation. I’m also replacing a bunch of LEFT JOIN parts of my queries to use RIGHT JOIN.

Another solution would have been to map that @OneToOne as a @OneToMany (basically a collection) and then I can use isEmpty() to check for null, and get(0) to get the value; however, this would still result in hibernate creating all kinds of lazy collection object that rarely get used.

Bookmark and Share

Filed under: hibernate, java, java ee, web development | Posted on October 7th, 2008 by Dobes | Comments

HQL: Be careful of using dot notation with left outer joins

I recently ran into a little glitch with my HQL that I thought I could share for the rest of you.

I created a query that looks something like:

SELECT b.c.d, a.b FROM a LEFT OUTER JOIN a.b AS b;

The problem with this is that hibernate will generate an inner join to resolve b.c.d, which PostgreSQL didn’t like; thus, in order to make this work I had to generate another outer join to access b.c:

SELECT c.d, a.b FROM a LEFT OUTER JOIN a.b AS b LEFT OUTER JOIN b.c AS c
Bookmark and Share

Filed under: database, java ee | Posted on October 3rd, 2008 by Dobes | Comments

Just when I thought the past would disappear

Google brings back the old search results from 2001.  Googling myself in 2001 I find the old version of dobesland.com, an old dream I had about the future of internet technology (now realized in the form of DHTML/AJAX and Cloud Computing, in a way).  Back then both of these sites were hosted on a cheap machine I stuck in my closet so the fan noise wouldn’t keep me awake, hooked up to my Cable internet connection.

What was your internet presence in 2001?

Bookmark and Share

Filed under: editorials | Posted on October 1st, 2008 by Dobes | Comments

Activity

Subscribe Me

TwitterCounter for @dobesv Share on FriendFeed

Popular Posts

Recent Comments

Blogroll

Tag Cloud

Categories

Archives

Copyright © 2009 Dobesland. All rights reserved.

Tech Blue designed by Hive Designs • Ported by Free WordPress Themes and Online Marketing