Saturday, December 19, 2020

Migration from GlassFish Application Server 3.1.2 to GlassFish Application Server 5.1 - EJB and Jersey Rest Services

GlassFish Application Server 3.1.2

Had used Enterprise Java Bean(EJB)s for developing Rest services using Jersey 1.x implementation and GlassFish Application Server 3.1.2.  Stateless EJBs were used using the annotations, and a sample EJB was declared as follows:

@Path("email/address")
@Local
@Stateless(name = "emailAddressProvider", mappedName = "ejb/emailAddressProvider")
public class EmailAddressProvider {

    . . .
    // REST methods
    . . .
}

This worked well without any issues, and was able to use call EJB services from other EJBs using the @EJB annotation.

GlassFish Application Server 5.1

GlassFish Applicaiton Server 5.1 supports Rest services implementation of Jersey 2.1.  While migrating the application from GlassFish Application Server 3.1.2 to 5.1, the code which worked well in GlassFish Application Server 3.1.2, failed to be deployed in GlassFish Applicaiton Server 5.1 with the message that the EJB EmailAddressProvider was not found while serving the Rest Services.  After a few trial and errors, and looking at the deployment of the application in the GlassFish Applicaiton Server it was observed that the EJB was named as EmailAddressProvider, and not emailAddressProvider.  Once the EJB name was changed to EmailAddressProvider,  the REST services were functional as expected.  The final sample code used for defining the REST services using EJBs was as follows:

@Path("email/address")
@Local
@Stateless(name = "EmailAddressProvider", mappedName = "ejb/emailAddressProvider")
public class EmailAddressProvider {

    . . .
    // REST methods
    . . .
}

 

Monday, January 21, 2019

Solr 7.6 - Solr Cloud installation

Solr 7.6 Getting Started

Solr documentation is huge.  In the time constrained age, of interest are the quick getting started.  As such the precise commands used to set up, and running SolrCloud7.6 with a three node external zookeeper ensemble is documented here.

Steps to create Solr Cloud:

  1. Install Zookeeper Ensemble
  2. Add a node
  3. Start Solr using the zookeeper configured in Step 1.
  4. Create ConfigSet
  5. Create Collection
  6. Access SolrCloud GUI

Install Zookeeper

Three node external zookeeper ensemble is used.  The three nodes used are:
  • wolf:2181
  • tiger:2181
  • lion:2181
The variable ZK_HOST would be wolf:2181,tiger2181,lion:2181

Add node

Adding a node to Solr Cloud provides detailed instructions to add a node to the cloud. 

Start Solr

The SolrCloud is also installed on the same nodes.

Start Solr using external zookeeper ensemble

<SOLR_HOME>/bin/solr start -c -z $ZK_HOST

Create ConfigSet

Without getting into detailed explanation of schema.xml, solrconfig.xml and other related configuration files, Sample Configuration files can be used to quickly get started.  Following script would generate a ConfigSet with the name tech, by uploading the configuration files stored in the configuration directory ~/solr/tech/.

#!/bin/sh

SOLR_HOME=~/solr-7.6.0
ZK_HOST=wolf:2181,tiger:2181,lion:2181
CONF_NAME=tech
CONF_DIR=~/solr/tech/

$SOLR_HOME/server/scripts/cloud-scripts/zkcli.sh -z $ZK_HOST -cmd upconfig -confname $CONF_NAME -confdir $CONF_DIR

Configure Zookeeper

Modify ZK_HOST in the file ~/SOLR_HOME/bin/solr.in.sh to read the ZK_HOST

ZK_HOST="wolf:2181,tiger:2181,lion:2181"

Create Collection

Following script can be used to create a new collection by the name mytech using the ConfigSet tech as created previsously with 3 shards per collection, and a replication factor of 2. 


#!/bin/sh



SOLR_HOME=~/solr-7.6.0
COLLECTION_NAME=mytech
CONFIG_NAME=tech
SHARD_COUNT=3
REPL_FACTOR=2
$SOLR_HOME/bin/solr create -c $COLLECTION_NAME -n $CONFIG_NAME -s $SHARD_COUNT -rf $REPL_FACTOR


Access SolrCloud GUI

The SolrCloud thus created can be accessed using the url http://wolf:8983/solr/.  The intutive user interface can be used to explore the various functionality provided by Solr.

Sample Data

Following sample data can be used to insert data from SolrCloud GUI:
id,cat,name,price,inStock,author,series_t,sequence_i,genre_s
0553573403,book,A Game of Thrones,7.99,true,George R.R. Martin,"A Song of Ice and Fire",1,fantasy
0553579908,book,A Clash of Kings,7.99,true,George R.R. Martin,"A Song of Ice and Fire",2,fantasy
055357342X,book,A Storm of Swords,7.99,true,George R.R. Martin,"A Song of Ice and Fire",3,fantasy
0553293354,book,Foundation,7.99,true,Isaac Asimov,Foundation Novels,1,scifi
0812521390,book,The Black Company,6.99,false,Glen Cook,The Chronicles of The Black Company,1,fantasy
0812550706,book,Ender's Game,6.99,true,Orson Scott Card,Ender,1,scifi
0441385532,book,Jhereg,7.95,false,Steven Brust,Vlad Taltos,1,fantasy
0380014300,book,Nine Princes In Amber,6.99,true,Roger Zelazny,the Chronicles of Amber,1,fantasy
0805080481,book,The Book of Three,5.99,true,Lloyd Alexander,The Chronicles of Prydain,1,fantasy

080508049X,book,The Black Cauldron,5.99,true,Lloyd Alexander,The Chronicles of Prydain,2,fantasy


Sample Query

Once the data has been added to the cloud, Solr search can be executed. Screenshot of Sample Query is shown below:


Solr 7.6 Creating Collection using ConfigSets, Errors


Solr7.6 Creating ConfigSet

Solr ConfigSets API provides rest end points for creating, listing, viewing, and deleting ConfigSets.  While trying to create a collection based on the ConfigSet created using the Rest Service as documented:
$ (cd solr/server/solr/configsets/sample_techproducts_configs/conf && zip -r - *) > myconfigset.zip

$ curl -X POST --header "Content-Type:application/octet-stream" --data-binary @myconfigset.zip "http://localhost:8983/solr/admin/configs?action=UPLOAD&name=myConfigSet"

results in the following error message:

ERROR: Failed to create collection 'tech' due to: {192.168.56.102:8983_solr=org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error from server at http://192.168.56.102:8983/solr: Error CREATEing SolrCore 'tech_shard3_replica_n10': Unable to create core [tech_shard3_replica_n10] Caused by: The configset for this collection was uploaded without any authentication in place, and this operation is not available for collections with untrusted configsets. To use this component, re-upload the configset after enabling authentication and authorization., 192.168.56.104:8983_solr=org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error from server at http://192.168.56.104:8983/solr: Error CREATEing SolrCore 'tech_shard1_replica_n2': Unable to create core [tech_shard1_replica_n2] Caused by: The configset for this collection was uploaded without any authentication in place, and this operation is not available for collections with untrusted configsets. To use this component, re-upload the configset after enabling authentication and authorization., 192.168.56.101:8983_solr=org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error from server at http://192.168.56.101:8983/solr: Error CREATEing SolrCore 'tech_shard2_replica_n4': Unable to create core [tech_shard2_replica_n4] Caused by: The configset for this collection was uploaded without any authentication in place, and this operation is not available for collections with untrusted configsets. To use this component, re-upload the configset after enabling authentication and authorization.}

The documentation says that functionality of uploading a configset is enabled by default, but can be disabled via a runtime parameter -Dconfigset.upload.enabled=false. Disabling this feature is advisable if you want to expose Solr installation to untrusted users (even though you should never do that!).

Instead of using the REST service for creating the configset, used the Solr's zkcli.sh file found under <SOLR_HOME>/server/scripts/cloud-scripts/, as shown below:


#!/bin/sh


SOLR_HOME=~/solr-7.6.0
ZK_HOST=wolf:2181,tiger:2181,lion:2181
CONF_NAME=tech
CONF_DIR=~/solr/tech/
$SOLR_HOME/server/scripts/cloud-scripts/zkcli.sh -z $ZK_HOST -cmd upconfig -confname $CONF_NAME -confdir $CONF_DIR


After executing the above command a Solr collection by the name tech as defined in the above script was successfully created.

Sunday, January 13, 2019

Delete all documents from Solr

How can I delete all documents from my index? documented on wiki shows the following commands which can be executed in the browser:
http://localhost:8983/solr/update?stream.body=<delete><query>*:*</query></delete>
http://localhost:8983/solr/update?stream.body=<commit/>
While trying to execute the same in SolrCloud 7.6, you get the message that stream.body is disabled.
RequestDispatcher shows the command to configure remote streaming.  Following is the command:
curl -H 'Content-type:application/json' -d '{"set-property": {"requestDispatcher.requestParsers.enableRemoteStreaming": true}, "set-property":{"requestDispatcher.requestParsers.enableStreamBody": true}}' http://localhost:8983/api/collections/

Wednesday, December 12, 2018

JPA object is not a known entity type

I have been using EclipseLink JPA for the last 7 years successfully.  Recently, while refactoring some of code I encountered the dreaded exception message "object is not a known entity type".  In the absence of a solution for the problem I settled to use the direct approach of using connection from DataSource and executing a PreparedStatement to create a record in the database.  Not giving up on the issue stumbled across the page: https://java-error-messages.blogspot.com/2011/05/is-not-known-entity-type.html.  Based on the explanation I revisited my code, and realized my mistake, and was able to fix the error.

Base class:
@XmlRootElement
@IdClass(ExtTestCostPK.class)
@MappedSuperclass
public class ExtTestCost implements Serializable {
. . .
. . .
. . .
}

Derived Class
@XmlRootElement
@Entity
@Cacheable(false)
@Inheritance(strategy=InheritanceType.TABLE_PER_CLASS)
@Table(name="V_EXT_TEST_COSTS")
public class EstlTestCost extends ExtTestCost {
. . .
. . .
. . .
}

Since I was refactoring the code, I was using following code to persist:
ExtTestCost extTestCost = new ExtTestCost();
// set the class attributes

entityManager.persist(extTestCost);

This piece of code was throwing "object is not a known entity type".  Having spent one sleepless night thinking how to solve the problem, after reading the link mentioned above, realizing the mistake that ExtTestCost indeed is not an entity, modified the code to read:
@XmlRootElement
@IdClass(ExtTestCostPK.class)
@MappedSuperclass
public abstract class ExtTestCost implements Serializable {
. . .
. . .
. . .
}

ExtTestCost extTestCost = new EstlTestCost();
// set the class attributes

entityManager.persist(extTestCost);

so that ExtTestCost can not be instantiated, and ExtTestCost was instantiated as EstlTextCost which indeed is an Entity.  Eclipselink JPA created a record in the database, and the exception disappeared.  Lesson learned is declare the base class abstract so that the default no argument constructor provided by the Java compiler can not be used, and you are forced to use a derived class.  

 

 

Wednesday, March 21, 2018

Tomcat Customization

Introduction

Recently I had a business requirement to intercept the raw request to a servlet container.  The normal entry point to a servlet through the ServletContext was inadequate.  Having worked with Tomcat for a long time, since its earlier version 3.0, I downloaded the source code, and the stepped through the code.  While stepping through the code I noticed that the method service(org.apache.coyote.Request req, org.apache.coyote.Response resof class org.apache.catalina.connector.CoyoteAdapter initiates the request processing and the raw request could be intercepted in this method, and hard coded and is instantiated in the class org.apache.catalina.connector.Connector.  While tomcat server configuration file conf/server.xml provides xml attributes for customization of Connector, could not figure out a way to customize CoyoteAdapter.  With these observations, I added a new parameter to the Connector class to dynamically instantiate the adapter class as against the hard coded CoyoteAdpater class.  With this dynamic instantiation of the AdapterClass I was able to achieve the required customization of Tomcat.  Details of the code changes are detailed in this blog.

Server Configuration


conf/server.xml is the configuration file for Tomcat.  The connector element as shown below is the one which provided the configuration parameters for Tomcat. 

<Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />

For dynamically instantiating the adapter, I added a new parameter called adapterClassName.  With the addition of the new parameter the Connector element changed to:


<Connector port="8080" protocol="HTTP/1.1"
               adapterClassName="<my custom adapter full class name>"
               connectionTimeout="20000"
               redirectPort="8443" />

Modifications to Connector

To read adapterClassName from the server configuration file added a new attribute 
adapterClassName to the Connector class, org.apache.catalina.connector.Connector and provided accessor and mutator methods.
To provide CoyoteAdpater as the default adapter and to dynamically instantiate the adapter based on the attribute specified in the server configuration, code related to adapter instantiation was changed from:
        adapter = new CoyoteAdapter(this);
to:
if(adapterClassName == null) {
adapter = new CoyoteAdapter(this);
} else {
        try {
Class<?> clazz = Class.forName(adapterClassName);
Constructor<Adapter> constructor = (Constructor<Adapter>) clazz.getConstructor(this.getClass());
adapter = (Adapter) constructor.newInstance(this);
} catch (ClassNotFoundException e1) {
e1.printStackTrace();
} catch (NoSuchMethodException e) {
e.printStackTrace();
} catch (SecurityException e) {
e.printStackTrace();
} catch (InstantiationException e) {
e.printStackTrace();
} catch (IllegalAccessException e) {
e.printStackTrace();
} catch (IllegalArgumentException e) {
e.printStackTrace();
} catch (InvocationTargetException e) {
e.printStackTrace();
}
}
As can be seen dynamic instantiation of custom adapter was no different than the existing instantiation except that reflection was used to instantiate the adapter.

Extending CoyoteAdpater

The class CoyoteAdpater is about 300 lines of code.  Instead of trying to rewrite the code, I went with the option of extending class and override the service method.

Compiling Tomcat

Since the custom adapter class is dynamically invoked the Tomcat's libraries are independent of the custom adapter and hence existing ant scripts in Tomcat's source code did not require any changes to compile the code.

Compiling Custom Adpater

Custom adpater class is dependent on several Tomcat libraries.  As such a separate Java project was created, with Tomcat libraries as dependencies.  A jar file containing the custom container was created, added to Tomcat lib folder.  Probably the jar file can be added to Tomcat's endorsed directory.  With these few changes I was able to see that custom adapter was instantiated and was working similar to CayoteAdapter.
The CustomAdpater can potentially be used for customizing Tomcat, and there could be several uses.  A few of them would be a proxy server for Microservices where microservice internal location could be determined from a variety of resources.

Summary

Intercepting raw http request in Tomcat requires Tomcat customization.  A simple solution using dynamic invocation was provided to intercept http raw request, and Tomcat customization was presented.  The minimal code changes required for Tomcat customization are presented.



Sunday, March 18, 2018

Integrating Microsoft Exchange Server with Web Applications in Java

Introduction

Recently in one of the projects I have been working I had a requirement to provide email address search where the backend email server was a Microsoft Exchange Server.  After trying several options, came across the open source java api, ews-java-api.  While getting started guide was pretty good, did not provide examples for searching the email addresses, which was my primary requirement.  Having searched the web for several hours then came across the commercial product, Jwebservices for Exchange.  This product provided a trial license for 30 days, and is relatively inexpensive compared with the hourly rates billed by developers.  The product has nice organized examples.  While trying the example code, it was observed that the class names in the commercial products were very similar to the open source API.  Based on the examples I was able to make some simple changes and use it in my web application for searching the email addresses.  Presented here is the sample code I was able to use within my application.

Sample Code

Following is the sample code I used:

import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;

import microsoft.exchange.webservices.data.core.ExchangeService;
import microsoft.exchange.webservices.data.core.enumeration.misc.ExchangeVersion;
import microsoft.exchange.webservices.data.credential.ExchangeCredentials;
import microsoft.exchange.webservices.data.credential.WebCredentials;
import microsoft.exchange.webservices.data.misc.NameResolution;
import microsoft.exchange.webservices.data.misc.NameResolutionCollection;
import microsoft.exchange.webservices.data.property.complex.EmailAddress;


public class MyExchangeService {

private ExchangeService service;

public MyExchangeService(String emailAddress, String password) {
service = new ExchangeService(ExchangeVersion.Exchange2010_SP2);
ExchangeCredentials credentials = new WebCredentials(emailAddress, password);
service.setCredentials(credentials);

try {
service.autodiscoverUrl(emailAddress);
} catch (Exception e) {
e.printStackTrace();
}
}

public List<EmailAddress> getEmailsAddress(String searchString) {
List<EmailAddress> emailAddresses = new ArrayList<>();
try {
NameResolutionCollection response = service.resolveName(searchString);
if(searchString.startsWith(". ")) {
Iterator<NameResolution> iterator = response.iterator();
while(iterator.hasNext()) {
NameResolution nameResolution = iterator.next();
emailAddresses.add(nameResolution.getMailbox());
}

} else {
Iterator<NameResolution> iterator = response.iterator();
while(iterator.hasNext()) {
NameResolution nameResolution = iterator.next();
emailAddresses.add(nameResolution.getMailbox());
}
}
} catch (Exception e) {
e.printStackTrace();
}

return emailAddresses;
}

public static void main(String[] args) {
final String EMAILADDRESS = "nonone@nowhere.none";
final String PASSWORD = "secret";

MyExchangeService myExchangeService = new MyExchangeService(EMAILADDRESS, PASSWORD);

String searchString = ". AC";
searchString = "somoone@somewhere.com";
List<EmailAddress> emailAddresses = myExchangeService.getEmailsAddress(searchString);

for(EmailAddress emailAddress : emailAddresses) {
System.out.println(emailAddress);
}
}
}

Performance

Compared with the commercial product the open source version is a little bit slower.  But for requirement the open source code worked.  I did not have to goto my project sponsor for $500/-