Author: Chandra Shekher Gupta

Technical Architect experience in Enterprise application/ Content/CMS and Big Data.

AEM form 6.2 OutPut Service to generate PDF using XDP Template

Sample JSP code to generate PDF using XDP Template and Data generated from Mobile HTML.

<%@include file=”/libs/foundation/global.jsp”%><%@include file=”/libs/foundation/global.jsp”%><%@ page session=”false” %><%@ page import=”org.apache.sling.api.resource.ResourceResolver”%><%@ page import=”java.io.*” %><%@ page import=”javax.xml.*” %><%@ page import=”org.w3c.dom.*” %>

<%
String UID = request.getParameter(“UID”);
Resource resTemplate= resourceResolver.getResource(“/content/dam/formsanddocuments/xxx-ria/Accounts_and_Advisors1.xdp/jcr:content/renditions/original”);javax.jcr.Node ntFileNodeTemplate = resTemplate.adaptTo(javax.jcr.Node.class);javax.jcr.Node ntResourceNodeTemplate = ntFileNodeTemplate.getNode(“jcr:content”);InputStream xdbis = ntResourceNodeTemplate.getProperty(“jcr:data”).getBinary().getStream();com.adobe.aemfd.docmanager.Document xdpDocument = new com.adobe.aemfd.docmanager.Document(xdbis);

Resource resData= resourceResolver.getResource(“/content/usergenerated/content/forms/XFA-xxx/Account/”+UID);javax.jcr.Node ntFileNodeData = resData.adaptTo(javax.jcr.Node.class);javax.jcr.Node ntResourceNodeData = ntFileNodeData.getNode(“jcr:content”);InputStream dataIs = ntResourceNodeData.getProperty(“jcr:data”).getBinary().getStream();com.adobe.aemfd.docmanager.Document dataDocument = new com.adobe.aemfd.docmanager.Document(dataIs);

com.adobe.fd.output.api.OutputService outputService = sling.getService(com.adobe.fd.output.api.OutputService.class);if(outputService ==null)                 {
System.out.println(“The output service is null..111…”);}com.adobe.fd.output.api.PDFOutputOptions pdfOptions = new com.adobe.fd.output.api.PDFOutputOptions();pdfOptions.setAcrobatVersion(com.adobe.fd.output.api.AcrobatVersion.Acrobat_11);com.adobe.aemfd.docmanager.Document generatedDocument = outputService.generatePDFOutput(xdpDocument,dataDocument,pdfOptions);//generatedDocument.copyToFile(new java.io.File(“c:/pg.pdf”));response.setContentType(“application/pdf”);byte[] bytes = com.google.common.io.ByteStreams.toByteArray(generatedDocument.getInputStream());//response.setContentLength(bytes.length);
response.getOutputStream().write(bytes, 0, bytes.length);response.getOutputStream().flush();System.out.println(“The output service is null..222222222…”);

%>

GoDaddy VPS Server shared hosted environment vs Dedicate Challenges Linux Box (Centos)

If you have pretty straightforward requirement of hosting PHP or Java Application requirement with MySQL, shared hosting environment is a good idea .

Shared environment has several limitation, one of those is iOS Notification port 2195 is’t  open. For that you need to switch to dedicated box.

Dedicated box exposes many limitation of GoDaddy marketing/tech support team. Godaddy charges you $375 for 120 GB HDD and 4 GB with no support includes any technical question. If you want that support , You would be charged additional close to $500 per year, creepy.

Go Daddy is changed a lot for past a year, they no more pick your call immediately, rather they put you in waiting queue for an hour or you drop a message for call back.

Generally, when you get dedicated VM created all software are pretty much lower version and you would face challenges to upgrade those . i.e. Curl version.

One of issue i encountered in my application needed curl 7.40+ while by default install was 7.19. I am not expert in Linux admin  and I had to struggle a lot to get that upgrade done..It was throwing many dependency error i tried to install 7.40+ curl RPM.

Finally i managed to do here is the step.

wget http://www.city-fan.org/ftp/contrib/yum-repo/city-fan.org-release-1-13.rhel6.noarch.rpm

rpm -ivh city-fan.org-release-1-13.rhel6.noarch.rpm

yum clean all

yum update cur

Hope this will help.

Thanks for your contribution Vicky Rihal

 

 

 

 

 

 

Spell Check Lucene in AEM 5.6.1

I am trying to implement Spell Check for one of content serach application when they do search since application is heavily based on JCR query using Query builder users pay for the content.

They need smart search features also “do you means this” features just like google.

Adobe has invested good amount of effort and and money to update/create the knowledge document but they still lacks in many module which are not core of the AEM.

Sales force connector is one the module which i believe has never worked with unlimited version of sales force. Not only me, I talked to couple of my friends they  promised customer to provide the Sales force integration out of box using sales force Template it works with Free SF edition but never with unlimited edition . Adobe support or Adobe forum users are clueless and end up writing custom Sales force rest API or WSDL to talk to AEM..that’s the one story.

Coming back to the Spell Check out of the box in AEM 5.6.1.. I wanted to give “Do mean this this Feature” to the search user and i enabled spellchecker module in workplace.xml.

<SearchIndex class=”com.day.crx.query.lucene.LuceneHandler”>
            <param name=”path” value=”${wsp.home}/index”/>
            <param name=”resultFetchSize” value=”50″/>
            <param name=”indexingConfiguration” value=”${wsp.home}/indexing_config.xml”/>
            <param name=”tikaConfigPath” value=”${wsp.home}/tika-config.xml”/>
            <param name=”supportHighlighting” value=”false”/>
            <param name=”spellCheckerClass” value=”com.day.crx.core.query.spell.CRXSpellChecker$OneMinuteRefreshInterval”/>
        </SearchIndex>

I ran the index which took couple of hours and then finally i got this created crx-quickstart\repository\workspaces\crx.default\index\spellchecker.

I wrote a Querybuilder code like this..

final QueryManager manager = session.getWorkspace().getQueryManager();
            Query query = manager.createQuery(“/jcr:root[rep:spellcheck(‘”+term+”‘)]/(rep:spellcheck())”, Query.XPATH);
            RowIterator rows = query.execute().getRows();
            // the above query will always return the root node no matter what string we check
            Row r = rows.nextRow();
            // get the result of the spell checking
            Value v = r.getValue(“rep:spellcheck()”);
            if (v == null) {
                termNew = term;
            } else {
                 termNew = v.getString();
            }          
        }
        catch(Exception ex){
            System.out.println(ex.getMessage() +”111″);
            log.error(“error caught in getSpelledChecked”,ex.getMessage());
        }
        System.out.println(” Source >> “+ term + ” Suggestion>> “+termNew);

And here the funny Output. If see some of the suggestion is good but learning comes as earning and manger comes as wagner :). This dictionary is not at all usable.

suggestion

Here is the most painful area.

Custom Spell Check Solution Did you means in AEM

Since Out of box solution in AEM for Spell Check seems not usable , I decided to use Lucene Spell Check API direly. rather CQ5 search APIw hich is wrapper on the of Lucene.

Here is the details of existing  CQ5 search bundle.
Bundle-Name: Social UGC Search Collections – Bundle which provides out of box spell check capabilities and lucene-core-3.6.1.jar is included in that bundle as supporting jar.

Here is the Custom Code

package com.xyz.util;
import java.io.File;
import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.search.spell.PlainTextDictionary;
import org.apache.lucene.search.spell.SpellChecker;
import org.apache.lucene.store.Directory;
import org.apache.lucene.store.FSDirectory;
import org.apache.lucene.util.Version;

public class Dictionary {
public static void main(String[] args) throws Exception {
File dir = new File(“B:/Projects/download/dic”);
Directory directory = FSDirectory.open(dir);
SpellChecker spellChecker = new SpellChecker(directory);
spellChecker.indexDictionary(new PlainTextDictionary(new File(“B:/Projects/dictionary/fulldictionary00.txt”)),
new IndexWriterConfig(Version.LUCENE_CURRENT,new StandardAnalyzer(Version.LUCENE_CURRENT)), false);
String wordForSuggestions = “mv money” ;
int suggestionsNumber = 1;
String[] suggestions = spellChecker.
suggestSimilar(wordForSuggestions, suggestionsNumber);
if (suggestions!=null && suggestions.length>0) {
for (String word : suggestions) {
System.out.println(“Did you mean:” + word);
}
}
else {
System.out.println(“No suggestions found for word:”+wordForSuggestions);
}
}
}

 

Dictionary sample

marketing on demand
compliancemax
enhanced trading
learning center
move money
compliance max

When you would deploy above code in AEM OSGI bundle, You would face org.apache.lucene.analysis.standard are not resolved. To resolve that you need to make changes in your maven
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
<configuration>
<instructions>
<Bundle-Category>xyz</Bundle-Category>
<Import-Package>
*
</Import-Package>
<Export-Package>
com.xyz.*,org.apache.lucene.*,org.tartarus.snowball.*
</Export-Package>
</instructions>
</configuration>
</plugin>

So now if you search mv money then suggestion would come like move money.

hope this will help.

 

 

 

Using GMail SMTP in AEM 6.0

smtp

If you are trying to use Gmail SMTP in AEM 6.0 or any Java based Mail Client and see above error logs.. authentication failed…not able to connect to smtp.gmail.com etc please make sure 2 things are done at your client machine where you are testing. i spend 4 to 5 hours to find the solution.

  1. Disable the firewall on your laptop or PC.
  2. Use following screenshot for SMTP configuration.

smtp1

Sample code here.

List<InternetAddress> emailRecipients = new ArrayList<InternetAddress>();
try {

@Reference
private MessageGatewayService messageGatewayService;
@Reference
private MessageGateway<HtmlEmail> messageGateway;
HtmlEmail email = new HtmlEmail();
emailRecipients.add(new InternetAddress(userName));
email.setTo(emailRecipients);
email.setSubject(“XXX Password Reset”);

StringBuffer sb = new StringBuffer();

sb.append(“<b>XXX Password Reset</b> <BR> To: “+userName);
sb.append(content);

email.setHtmlMsg(sb.toString());
messageGateway = messageGatewayService.getGateway(HtmlEmail.class);
messageGateway.send(email);
} catch (Exception ex) {
ex.printStackTrace();
LOGGER.debug(” Exception caught in send email” + ex.getMessage());

return false;
}

 

 

User authentication using AngularJS, PHP, MySQL

I appreciate Swadesh Behera for contributing so much to Web Community who provides several  demo application on Angular JS, PHP and mySQL.

Particularly http://www.angularcode.com/user-authentication-using-angularjs-php-mysql/ is great.

I did setup on my local windows 2007 Laptop with PHP Separate distribution using Windows Apache 2.4.

People have consistently faced problem when they are running the Demo application in their local env.

session, signup and login HTTP Get and Post services consistently failed with 404 error.

I faced too and understood that issue would be primarily by doing two things.

  1. .htaccess
  2. httpd.conf allowing URL so 404 error could be resolved.

Steps i did to fix the issue.

  1. .htaccess

    RewriteEngine On
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteRule ^(.*)$ %{ENV:BASE}index.php [QSA,L]

  2. httpd.conf.. Allowoveride ALL has to be done to fix 404 error.
    <Directory “Z:/Projects/xyz/Apache24/htdocs”>
    #
    # Possible values for the Options directive are “None”, “All”,
    # or any combination of:
    #   Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews
    #
    # Note that “MultiViews” must be named *explicitly* — “Options All”
    # doesn’t give it to you.
    #
    # The Options directive is both complicated and important.  Please see
    # http://httpd.apache.org/docs/2.4/mod/core.html#options
    # for more information.
    #
    Options Indexes FollowSymLinks

    #
    # AllowOverride controls what directives may be placed in .htaccess files.
    # It can be “All”, “None”, or any combination of the keywords:
    #   AllowOverride FileInfo AuthConfig Limit
    #

        AllowOverride All

    #
    # Controls who can get stuff from this server.
    #
    Require all granted
    </Directory>

 

 

 

 

 

 

Static Code Analysis Custom Check style with Maven build CQ5/AEM

Any Maven Build tool can use Check style for code analysis.

List of items to be touched.

<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-checkstyle-plugin</artifactId>
<version>2.9.1</version>
</plugin>
</plugins>
</pluginManagement>
</build>

  • You are done with your changes. Run your command like “mvn clean install checkstyle:checkstyle” since you provided your config for google_checks…it will use the same if you remove the properties it will use out of box and your code analysis report would be created under /target/site as html and XML.

Please let me know if you have any question.

AEM 5.6.1 Reducing Repository size and Nailing down Web Site Navigation slowness

If CQ5/AEM 5.6 or 5.6.1 is being used as brochure style heavy content specific solution i.e. digital platform where number of content are like 50 k or more, CRX repository file size would be like 100 GB or more over the year.

You may face slowness when content author are accessing website console or you may find CRX repo size 5or 10 times bigger than what it is expected. Adobe provides/recommend repository optimization  as routine jobs weekly or bi-weekly to keep check on system health.

https://helpx.adobe.com/experience-manager/kb/performancetuningtips.html.

Problem We faced in our environment.

  1. Production is non-cluster in Author and Publish both. Author instance  ideal size was reduced to 25 GB from 125 GB after series of optimization overnight.
  2. Content author were having slowness in web site console  navigation when node has sizable content on first level i .e 10+ node.

That slowness was not related with repository size by the way. One of the publish agent configuration content was causing that slowness to prove it when agent was disabled slowness disappeared.
Steps to reduce Repository.

1. Set CRX Search Index’s resultFetchSize & cacheSize param

 

C:\CQ5-Author\crx-quickstart\repository\workspaces\crx.default\workspace.xml 
<SearchIndex class="com.day.crx.query.lucene.LuceneHandler">
<param name="path" value="${wsp.home}/index"/> 
<param name="resultFetchSize" value="50"/>
<param name="cacheSize" value="100000" />
</SearchIndex

2. Cache Tar PM index

C:\CQ5-Author\crx-quickstart\repository\workspaces\crx.default\workspace.xml 
<PersistenceManager class="com.day.crx.persistence.tar.TarPersistenceManager">
<param name="indexInMemory" value="true" />
</PersistenceManager>

3. disable clustering

C:\CQ5-Author\crx-quickstart\repository\repository.xml
<Cluster>
<Journal class="com.day.crx.persistence.tar.TarJournal"/>
</Cluster>

Once above configuration related changes is completed then perform these activity provided in AEM from system/console JMX tools.

1.Purge workflow instance data /system/console/jmx and then choose com.adobe.granite.workflow “Maintenance” and choose purgeCompletedWorkflow.
2.Perform Tar Index merge it might take 10 to 20 minutes…please check error.log  choose repository under /system/console/jmx and tarIndexMerge.
3.Perform tar optimization manually that might go overnight under repository under /system/console/jmx by clicking “startTarOptimization“.
4.Perform Step 2 again that might go for 10 to 20 minutes again.
5.Finally run DataStoreGarbageCollection under repository /system/console/jmx… and finally you will see disk space used is reclaimed.


 

		

Middle level technology solution company challenge: Embracing Enterprise Application Framework

I worked in start up open source technology company for 7 years and was amazing journey in building platform using Apache ServiceMix,  Lifreay Portal, JBoss middle ware suites, Alfresco and many others as  integrated solution to achieve business need for large banking, social care  in Africa, Europe and Indian market.

I recently had three days in-house TIBCO Training includes TIBCO Business works, EMS, Designer, Active Space , spot fire and other tools related with deployment , Continuous integration and logging framework. Team did great jobs in terms of delivering basic building blocks and architectural nuts and bolts which are required to develop service Web Service, Rest Full Service, querying Database using TIBCO Designer no custom code, it’s all ready made TIBCO provided pallets and Hops and I simply said wow because I am coming from ETL background where ready-made hops and tools saves developers time and reduces risk of long and buggy code for file reading/transformation/querying Database and many more.

Here is one of the sample service designs in TIBCO designer to read some data from database (No Java/.net class no custom code)

TIBCO Designer

As i software solution architect I am convinced that developing a service is much easier with TIBCO than  writing Custom Java or .Net code and writing JSON and other transformation code, it helps to reduce buggy code and faster turnaround time for solution. It can’t be denied Initial development time would be higher for those who are new to this.

NoSQL/Data Grid (Active Space) in TIBCO

I always built robust /scalable low-cost solution based on liferay, alfresco on JBoss and many more .. no software license but supporting thousand concurrent public users. Map/reduce, Lucene, Indexing , SOLR are basic ingredient the solution we developed. I am keen follower of this scalable platform such as Amazon, Netflix , E-Bay, price-line etc .

Active space is new addition to TIBCO suite to support high number of concurrent users in competition of NoSQL/Data-grid/Cache technology which TIBCO claims runs on commodity server, but in Demo I found they are running two nodes on 512 GB memory which are beyond my thoughts why they solution in such a way, later I realized its in memory non-persistent solution where data resides in RAM not on Disk.

Challenge in Adoption Non Relational Approach?

I got a chance to speak few architects who have developed financial platform Microsoft .net for more than decades and very often I hear in leadership meeting reducing downtime , upgrade etc. I always ask question to myself why can’t we move to new generation database  and here is reply

“ This new technology and framework are bad adoption and does not fit in our solution, we are very happy with our existing solution and we can scale in this only. New solution are very slow , no documentation so not acceptable blah blah blah”.

Fitting Everything together?

One most important philosophies underpin that one size does not fit all, for many year traditional Database server are used for storing all types, it doesn’t matter all data types fits for relational Data model. General perception is that media, Analytic and social media application are suitable for NoSQL Database and structured relations data but that’s not correct.

Following can be implemented in exciting software to scale and reduce downtime.

  1. Memcached is used by Both Amazon and Netflix for frequently  requested content/data same can be achieved by Active Space to reduce load on database and faster response to user requests.
  2. Let us stick using highly secure data in RDBMS but active/active set up can be done to reduce downtime and scale it.
  3. Move other kind of information for example user profile and other data in distributed  NoSQL Database and redesign the service.

 

Profile based Deployment using Maven In AEM (CQ5)

This presume Maven skeleton has been already created.

  • UI
  • Bundle
  • root pom.xml files.

Update/create  this entries specific to your environment in root pom.xml

Step 1.

<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<!– Development/Integration Server properties –>
<!– Author –>
<cq.devint.author.host>10.29.136.192</cq.devint.author.host>
<cq.devint.author.port>4502</cq.devint.author.port>
<cq.devint.author.username>admin</cq.devint.author.username>
<cq.devint.author.password>admin</cq.devint.author.password>
<!– Publish –>
<cq.devint.publish.host>10.29.136.194</cq.devint.publish.host>
<cq.devint.publish.port>4503</cq.devint.publish.port>
<cq.devint.publish.username>admin</cq.devint.publish.username>
<cq.devint.publish.password>admin</cq.devint.publish.password>
</properties>

Step2 .

Create below profile entry in same pom.xml or where ever you want.

<profile>
<id>dev-int</id>
<build>
<plugins>
<plugin>
<groupId>com.day.jcr.vault</groupId>
<artifactId>maven-vault-plugin</artifactId>
<configuration>
<failOnError>true</failOnError>
</configuration>
<executions>
<execution>
<id>install-package-author</id>
<goals>
<goal>install</goal>
</goals>
<configuration>
<packageFile>${project.build.directory}/${project.build.finalName}.zip</packageFile>
<targetURL>http://${cq.devint.author.host}:${cq.devint.author.port}/crx/packmgr/service.jsp</targetURL>
<userId>${cq.devint.author.username}</userId>
<password>${cq.devint.author.password}</password>
</configuration>
</execution>
<execution>
<id>install-package-publish</id>
<goals>
<goal>install</goal>
</goals>
<configuration>
<packageFile>${project.build.directory}/${project.build.finalName}.zip</packageFile>
<targetURL>http://${cq.devint.publish.host}:${cq.devint.publish.port}/crx/packmgr/service.jsp</targetURL>
<userId>${cq.devint.publish.username}</userId>
<password>${cq.devint.publish.password}</password>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>

and finally run this command from command line or any CI Server profile.

mvn clean install -P dev-int