20th Aug2010

Lusine – Two Dots

by admin

The album this comes from is well worth checking out, A Certain Distance, but for me this is the standout track and a lovely soundtrack to a sunny summer’s day night.   Can’t beat tracks like this and been following up on some of the back catalogue and remixes of Lusine’s since.  Enjoy and as always, play it loud.

20th Aug2010

Pushing large data sets using LCDS Cluster and JGroups

by admin

One of my colleagues, Harpreet Singh, was working on an issue with LiveCycle Data Services where a large server side DataServiceTransaction which contained around 500 objects created and pushed out to connected clients would stall when getting transferred across the JGroups cluster.

Harpreet found that using the default jgroups-tcp.xml supplied with LCDS, there is a setting which controls the maximum bundle size in JGroups which defaults to 64000 if not present in the configuration.   By adding the max_bundle_size  attribute and upping the default size in the TCP tag in the jgroups configuration, LCDS was able to push the large data set across the clustered instances.

The actual exception getting thrown (which isn’t output in JGroups 2.7) is similar to the following:

 

TCP:896 – failed sending message to null (127879 bytes): java.lang.Exception: message size (127879) is greater than max bundling size (64000). Set the fragmentation/bundle size in FRAG and TP correctly

 

So if you find some data not getting pushed across your LCDS cluster, check this setting.  Great work from Harpreet.

20th Aug2010

fabric 52: Optimo (Espacio)

by admin

I’ve had this a while, and while I wasn’t sure the first few listens, I’m now convinced that this is indeed a classic.  Buy electronically at Bleep (who kindly supply the player) or in your local (indepedent!) record shop.

If you like this, I would suggest checking out their Acid podcast Acid Eyeful, another classic and lovely play of wording on that Laurent Garnier classic.

18th Aug2010

Controlling serialization of Java Objects in LCDS or BlazeDS

by admin

I was working on a problem, where the JPA provider of WebLogic 11g (EclipseLink) was being destroyed when an OptimisticLockException being thrown by EclipseLink was being wrapped in a MessageException to be thrown by LiveCycle Data Services (or BlazeDS).  

After some investigation and debugging the MessageException.cause property, which is set through Throwable.initCause(), showed the following object graph.  As detailed below, when using EclipseLink, it has it’s own OptimisticLockException, which is then wrapped by the JPA version and thrown:

  • MessageException  (id=104)
    • cause EJBException  (id=124)
      • cause javax.persistence.OptimisticLockException  (id=130)
        • cause org.eclipse.persistence.exceptions.OptimisticLockException  (id=137)

 

Cause and Effect

When this MessageException is thrown by the MessageBroker, it travels up the stack and eventually gets serialized to be transferred across to the Flex client.   Somewhere in this serialization, the JPA session seems to get destroyed and any subsequent requests to the server from the client get the following error in the FaultEvent to the Flex client:

  • javax.ejb.EJBTransactionRolledbackException : EJB Exception: ; nested exception is: java.lang.NullPointerException; nested exception is: java.lang.NullPointerException
Looking at the stack trace of the server logs, it gives a rather strange error, of which I couldn’t find out much about through our friend Google:
  • java.lang.NullPointerException at org.eclipse.persistence.internal.sessions.AbstractRecord.get(AbstractRecord.java:270)

Reasoning and Proof

I was able to, first, isolate the error from the BlazeDS application I was using into a simple servlet, simply by updating a Managed Entity twice in some Java without refreshing from the database.   When I got an OptimisticLockException in the Java, I used the Narkisr technique to perform serialization on the OptimisticLockException thrown, and lo and behold, the JPA session appeared to be destroyed, and the whole application rendered unusable until a server restart.

So far, this showed me a few things:

  1. When the application was in a “ruined” state, it seemed to have an open JPA session or such like that then locked any other requests/transactions from the client
  2. It wasn’t BlazeDS or LCDS per se that is the problem, but in the serialization of EclipseLink OptimisticLockException to AMF that caused the problem.
  3. Since it is BlazeDS code that is doing the serialization, then I could head on over to the BlazeDS repository for it and download the code for it!

Solutions and Dilutions

So that’s how I got to the title of the post, how to control serialization of Java Objects in LCDS or BlazeDS.   There are a few posts out there on the wild web that show how to do this, and I am borrowing heavily from them but it makes sense for me to show one of my solutions (yes, there was another ;) ) in the context of this post.

From debugging the exception, I could see the full Object graph in Eclipse.   Whilst I knew we wanted to throw the javax.persistence.OptimisticLock exception across, as well as the EJBException from the bean that was throwing the error, I also knew that if we serialized the org.eclipse.persistence.exceptions.OptimisticLockException then we would then get the application into a state that we were unable to recover from, other than restarting the server, not a good solution.

The bits of the puzzle that needed to be brought together to investigate and find a way to tell the Data Service not to serialize the org.eclipse.persistence.exceptions.OptimisticLockException are:

As always, someone has always said it better than me, and none more so than Peter Farland over on this Adobe.com forum post in response to some questions.   I’m posting it here for context (it contains some extraneous info to this blog post about cloning, but still gold info):

PropertyProxy’s play two roles – the first is to control how EVERY instance of a complex type should be described for serialization, the second is to control how just this ONE instance of a complex type should be described for serialization. 

The PropertyProxyRegistry simply maps a type (class or interface) to an instance of some PropertyProxy implementation. This serves as a reusable “template” that can be used for EVERY instance of a complex type as they pass through the serializer. (This avoids the wasteful creation of new PropertyProxy’s just to introspect an instance of a complex type). 

However, perhaps you have a special requirement for just ONE instance of your complex type. In this scenario you look up the appropriate PropertyProxy for the complex type – but you need to make some specific adjustment say, excluding some property but just this once, for your value. In this scenario the PropertyProxy acts as a wrapper and the value serves as the default instance of the PropertyProxy. The serializer will see the value is wrapped in a PropertyProxy and will not consult the PropertyProxyRegistry. 

Now, the problem is that you don’t want to modify the template PropertyProxy instance in the registry as that would impact EVERY other instance. So, you can clone the PropertyProxy for your ONE instance and make the adjustment. 

If you didn’t care about the state of the template PropertyProxy in the registry, you could also simply construct your own PropertyProxy instance and use that to wrap your value. 

Since the template PropertyProxy instances are shared across threads, then they should not be modified in an unsafe manner. Generally, however, it’s not a concern as template PropertyProxy instances are not modified after construction and the value to be introspected is passed in as an argument to the utility methods (rather than being part of the PropertyProxy state).

So the simple solution in in the first instance, was to create a PropertyProxy for the javax.persistence.OptimisticLockException, and make sure that it did not serialize the cause property (look up remember the object graph!).  

I then used the technique to eliminate the whole org.eclipse.persistence.exceptions.OptimisticLockException from the serialization process, but through trial and error, I later found that it was specifically serialization of the  session property and the query property of the exception.   I then scoured by eyeball to check for any other appearances of these properties to guard against serialization of them into AMF.

So, my final three bits of code ended up looking like the following:

EclipseLinkExceptionsProxy.java

 package com.adobe.trt.lcds;

import flex.messaging.io.BeanProxy;

public class EclipseLinkExceptionsProxy extends BeanProxy {
 private static final long serialVersionUID = 3809660049108090824L;
 
    static
    {
     addIgnoreProperty(org.eclipse.persistence.exceptions.EclipseLinkException.class, “session”);
     addIgnoreProperty(org.eclipse.persistence.exceptions.DatabaseException.class, “query”);
     addIgnoreProperty(org.eclipse.persistence.exceptions.OptimisticLockException.class, “query”);
    }
}

EclipseLinkExceptionsProxyBootstrapService.java

package com.adobe.trt.lcds;

import flex.messaging.config.ConfigMap;
import flex.messaging.io.PropertyProxyRegistry;
import flex.messaging.services.AbstractBootstrapService;

public class EclipseLinkExceptionsBootstrapService extends AbstractBootstrapService {

 @Override
 public void initialize(String id, ConfigMap properties) {
  PropertyProxyRegistry registry = PropertyProxyRegistry.getRegistry();
  registry.register(Object.class, new EclipseLinkExceptionsProxy());
 }

 @Override
 public void start() {
  //No-op
 }

 @Override
 public void stop() {
  //No-op
 }
}

services-config.xml for the BootstrapService

    <services>
        <service-include file-path="remoting-config.xml" />
        <service-include file-path="proxy-config.xml" />
        <service-include file-path="messaging-config.xml" />

        <service id="eclipselinkExceptionsProxyBootstrap" class="com.adobe.trt.lcds.EclipseLinkExceptionsProxyBootstrapService"/>
    </services>

 The application could then throw an OptimisticLockException and continue as expected.

Footnotes

This was only tested against the OptimisticLockException issue, but it is a good start into how to control the Serialization of your Java Objects in LCDS or BlazeDS.  I would definitely recommend a good look about at all the classes that implement the PropertyProxy class, you can get a lot of good tips from how to do things in there.

17th Aug2010

Surpressing values in debug output of BlazeDS / LiveCycle Data Services

by admin

The best documentation for the configuration files for LiveCycle Data Services are found after you install the product.   If you look under <install>/resources/config you will find a set of config files with explanations for each settings.  For instance here are the LCDS 3.1 configuration file set.

Sometimes there are some hidden gems in them, and one of the ones I came across again today was on surpressing values in your debug output when you have the logging level set to Debug.   By including something similar to below, you can ensure that your log files don’t accidentally record some confidential or important information:

In services-config.xml   

     <logging … level=”Debug”>
          …
         <properties>
            <exclude-properties>
                <property>oldPassword</property>
                <property>password</property>
                <property>session</property>
            </exclude-properties>
        </properties>
    </logging>

 

What that will get you when you look at your debug output, for example for any property called ‘session’ is:

     session = ** [Value Suppressed] **

Result all round, your logs don’t contain information that you don’t want to either share or see!

16th Aug2010

Oracle 11g R2 Certified with ColdFusion 9.0.1

by admin

Oracle 11g R2 is now certified to use with ColdFusion 9.0.1.   This was enabled by the shipping of the Data Direct 4.1 drivers with CF9.0.1 (whereas it was 4.0 with CF 9.0.0).   Enjoy.

03rd Aug2010

CFHttpProtocolHandler exception when starting ColdFusion

by admin

If you get the following error when starting ColdFusion 9, then you are most likely trying to install LCDS 3.x into ColdFusion 9.0.0, you need ColdFusion 9.0.1 for that.

  • can’t find the class “coldfusion/flash/messaging/CFHttpProtocolHandler”

I name my EAR’s cfXXX.cfusion.ear, and my cf901 one wasn’t really CF 9.0.1!   I checked everything, then I checked the plug :)

03rd Aug2010

Model Driven Development and Overriding the Default JPA Annotations

by admin

LiveCycle Data Services 3 comes with a new feature, Model Driven Development.   This new feature looks very powerful and folk are finding their feet with it and I’ve seen some really cool stuff.   I’m going to call it Fiber though, as that was its codename. Fiber lets you define your Entities using a Eclipse plugin where you can visualise the relationship between Entities and specify properties and attributes.

When generating your code using the Fiber plugin (make sure and get at least version 1.0.2), the Java and Flex code splits up to give two classes for every entity.  Say we had a Customer entity, you would get the following classes generated:

  • _Super_Customer.java
  • Customer.java
  • _Super_Customer.as
  • Customer.as

Good practice dictates that the “_Super_” class should never be altered, as any changes will be overwritten every time you generate/deploy the model to the server.   Now, the “” class is where you can put your own code, safe in the knowledge that it won’t be lost.   Fiber uses JPA Annotations on the Java side in the “_Super_.java” class to define the relationship between the entities.

I’ve give an example of each Java class as produced by default by Fiber:

_Super_LineItemBean.java

/** * This is a generated class and is not intended for modification.  To customize behavior * of this value object you may modify the generated sub-class of this class - LineItemBean.java. */

package services;

import services.InventoryBean;import services.MaintainOrderBean;import java.io.Serializable;import javax.persistence.*;

@MappedSuperclasspublic class _Super_LineItemBean implements Serializable{     ... snipped ...

    @JoinColumn(name="ORDER_NUMBER")    @org.hibernate.annotations.ForeignKey(name="FK_LineItemBean_orderBean_TO_MaintainOrderBean")    @ManyToOne(fetch=FetchType.LAZY)    public MaintainOrderBean getOrderBean()    {        return _orderBean;    }

    ... snipped ...}

LineItemBean.java

package services;

import javax.persistence.*;

@Entity@IdClass(LineItemBeanPK.class)@Table(name="ITEMS")public class LineItemBean extends _Super_LineItemBean{

}

Fiber gives you some options in the Eclipse Plug in to configure these relationships (shown by the @JoinColumn annotation in the _Super_.java class above) but sometimes that isn’t enough.  Well, you can log a wish request that Fiber will supports your request in a future version, but you can also override the JPA annotation in the .java class.   The _Super_.java uses @MappedClass, defined as:

Designates a class whose mapping information is applied
 to the entities that inherit from it. A mapped superclass
 has no separate table defined for it.

It’s the actual .java class that triggers a persistance table, and from the documentation for @MappedClass, it also exposes some extra annotations which allow you to override the @MappedClass defined relationships:

So, say we wanted to add some attributes to the @JoinColumn(“ORDER_NUMBER”) defined in the _Super_LineItemBean.java, we would alter the LineItemBean.java to be as follows:

Customised LineItemBean.java

package services;

import javax.persistence.*;

@Entity@IdClass(LineItemBeanPK.class)@Table(name="ITEMS")@AssociationOverride(name="ORDER_NUMBER", joinColumns=@JoinColumn(name="ORDER_NUMBER",insertable=false,updatable=false))public class LineItemBean extends _Super_LineItemBean{

}

And then we have the JPA annotations as we need, without the danger that they could be overwritten by Fiber.  Go check out the documenation and samples for more info on it and stuff to play with.

01st Aug2010

Charles HTTP Proxy and SSL

by admin

Internally we use Charles HTTP Proxy for everything (well, not RTMP but one can wish for that).   It’s usually one of the first things I ask for when troubleshooting web related issues.  Often SSL is involved, and folk send in Charles trace’s that haven’t captured the SSL traffic and the session is just garbled text.

Usually the reason for this is that the Charles CA hasn’t been installed, which allows Charles to capture the HTTP traffic, decrypt it, log it, and then reencrypt it so that the SSL request/response is kept end to end.

I’ve given these instructions out a few times, so thought would capture them here.  They are for Windows, but the same concepts apply to other OSes.

1.  Install Charles CA from the Help Menu

Install Charles CA SSL Certificate

 

2. Choose to install Certificate from Windows Dialog

Windows Certificate Installation Window

 

3. Choose to place the Cert in a specified location, I always place it in the Trusted Root CA store.

Importing the CA Certificate as Trusted

 

4. Remember and add the site you want Charles to intercept in the Proxy Settings > SSL within Charles.

Adding a site to the SSL Proxying List

 

Charles should now capture the SSL traffic and show meaningful traces.  One caveat, I’ve also seen this cause problems when load testing through Charles, so if you get strange results, always try without Charles too.